Reindex - application for visualize, optimize and automate your Elasticsearch reindex process

Overview

Welcome to reindex application for Elasticsearch

This application will help you to reindex one or more existing indices, into the local or remote Elasticsearch cluster.

Complete guide can be found on our web site: https://dbeast.co/#/reindex-for-elasticsearch.

Download the application: https://github.com/dbeast-co/Reindex/releases/.

The reindex destinations:

  • Merge multiple indices into one index
  • Reindex to alias
  • Reindex to the index named with prefix + original_index_name
  • Reindex to the index named with original_index_name + suffix
  • Reindex to ILM rollover alias with the possibility to create first ILM index (for example index-000001 for size rollover or %3CINDEX_NAME-%7Bnow%2Fd%7D-000001%3E for time series indices)
  • Remote reindex with the same index name

Features:

  • Reindex support SSL with a certificate (you'll need to upload it), or without (we will set the SSL verification to "none")

  • Reindex doesn't save your passwords. In case of the restart the application you'll need to insert them again.

  • Reindex work with two reindex algorithms:

    1. Whole index algorithm - regular reindex, that reindex index as is, with the one reindex request
    2. Time oriented algorithm - each index will be splatted into several time frames, and each time frame will be run, as a separated reindex request ("reindex by query" with the "range" query). You have to define the time frame - the period of the data chunk in minutes, the field, that will be used as a date field and format of this field
  • You can set up the number of concurrently processed indices, and the number of concurrent data frames per index (in case of Time oriented algorithm usage)

Requirements

  • Java 8+

  • For remote reindex, you'll need to define the reindex.whitelist setting in your elasticsearch.yml file in each Elasticsearch node of the remote cluster, you'll require to define: reindex.remote.whitelist: "otherhost:9200, another:9200, 127.0.10.:9200, localhost:" (PAY ATTENTION! There is no http/https at start of the address)

  • For remote reindex to cluster with https connection: In your elasticsearch.yml file in each Elasticsearch node of the remote cluster you'll require to define: reindex.remote.whitelist: "otherhost:9200, another:9200, 127.0.10.:9200, localhost:" (PAY ATTENTION! There is no http/https at start of the address) reindex.ssl.verification_mode: none (One of full, none, certificate) reindex.ssl.certificate: /etc/elasticsearch/certs/ca.crt (Path to root ca) This is only example... you can use another SSL settings

Installation

This application works via a web browser, so you could use it on the local or remote machine

Installation steps

Unpack the reindex.zip file in your local or remote machine

For the Windows

  1. Edit file config/reindex.yml Specify the host and port host: HOST_OR_IP port: PORT

  2. Run file reindex.bat

For the Linux

  1. From the application home folder: chmod +x reindex.sh

  2. Edit file config/reindex.yml Specify the host and port host: HOST_OR_IP port: PORT

  3. ./reindex.sh

Known issues

  1. The application tested only in the Chrome browser
  2. In Windows, in the time of application running, if you click within the console, it will pause the output. This could happen, because of QuickEdit Mode and/or Insert options are checked in the console settings. To get to these settings, right-click on the PowerShell/Console-Logo in the top-left of your terminal window, then select 'Properties'

Questions... Feedbacks... Suggestions...

If you'll find any bugs, or have any questions/suggestions/feedbacks, please feel free to open issues, start discussion or mail us: [email protected]

Terms and conditions

Copyright © 2021 Matskeplishvili Vakhtang https://dbeast.co. Licensed under the Apache License, Version 2.0

Comments
  • Bump jackson-databind from 2.11.2 to 2.12.7.1

    Bump jackson-databind from 2.11.2 to 2.12.7.1

    Bumps jackson-databind from 2.11.2 to 2.12.7.1.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies java 
    opened by dependabot[bot] 1
  • Bump jackson-databind from 2.11.2 to 2.13.4.1

    Bump jackson-databind from 2.11.2 to 2.13.4.1

    Bumps jackson-databind from 2.11.2 to 2.13.4.1.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies java 
    opened by dependabot[bot] 1
  • Bump log4j-core from 2.17.0 to 2.17.1

    Bump log4j-core from 2.17.0 to 2.17.1

    Bumps log4j-core from 2.17.0 to 2.17.1.

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies java 
    opened by dependabot[bot] 1
  • Bump jackson-databind from 2.11.2 to 2.12.6.1

    Bump jackson-databind from 2.11.2 to 2.12.6.1

    Bumps jackson-databind from 2.11.2 to 2.12.6.1.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies java 
    opened by dependabot[bot] 1
  • [UI] In the remote reindex

    [UI] In the remote reindex "use same index name" doesn't allow to save the project

    Describe the bug A clear and concise description of what the bug is.

    To Reproduce Steps to reproduce the behavior:

    1. Go to '...'
    2. Click on '....'
    3. Scroll down to '....'
    4. See error

    Expected behavior A clear and concise description of what you expected to happen.

    Screenshots If applicable, add screenshots to help explain your problem.

    System (please complete the following information):

    • OS: [e.g. iOS]
    • Browser [e.g. chrome, safari]
    • Elasticsearch Version [e.g. 7.15.1]
    • Version [e.g. 22]

    Additional context Add any other context about the problem here.

    bug 
    opened by tankha 0
  • [QA] Test the application behaivior with different date types

    [QA] Test the application behaivior with different date types

    Need to test the application behavior in case of "Time oriented" algorithm, when the date field have more than one date format in the documents. For example: one document have "YYYY.MM.DDThh:mm:ss" and the second one have "YYYY.MM.DDThh"

    Test 
    opened by dbeast-co 0
  • [SERVER] Speed up the reindex process with remove refresh_interval

    [SERVER] Speed up the reindex process with remove refresh_interval

    Reindex to the index with default refresh_interval (null) could speedup reindex speed and decrease CPU usage. Could be implemented as automation for the following steps:

    1. Start reindex
    2. Check the refresh_interval in the destintaion index settings
    3. If refresh_interval exists -> set refresh_interval to null
    4. After the end of the proccess return the refresh_interval to original value
    enhancement 
    opened by dbeast-co 0
  • [SERVER] Speed up the reindex process with disable replication

    [SERVER] Speed up the reindex process with disable replication

    Reindex to the index without replication could speedup reindex speed and decrease CPU usage. Could be implemented as automation for the following steps:

    1. Start reindex
    2. Test number of replica
    3. If number of replica more then 0 -> set replica to 0
    4. After the end of the proccess return the number of replicas to original value

    Maybe add this option to the UI.

    enhancement 
    opened by dbeast-co 0
Releases(Release)
Owner
DBeast
DBeast
👾 A React Native app to visualize your NFTs in AR

AR NFT ?? Description AR NFT is a mobile app developed using React Native, Wallet Connect, and Viro React to allow users to login in your favorite wal

Akinn Rosa 79 Nov 11, 2022
This application will help you to generate Elasticsearch template based on your data

Welcome to templates generator application for Elasticsearch This application will help you to generate the template and/or test index, based on your

DBeast 2 Jan 2, 2023
Kyrestia, named after Kyrestia the Firstborne, is a process engine supporting mainstream process definition standards.

Kyrestia Kyrestia, named after Kyrestia the Firstborne, is a process engine supporting mainstream process definition standards. It is not only lightwe

Weiran Wu 32 Feb 22, 2022
mall学习教程,架构、业务、技术要点全方位解析。mall项目(40k+star)是一套电商系统,使用现阶段主流技术实现。涵盖了SpringBoot 2.3.0、MyBatis 3.4.6、Elasticsearch 7.6.2、RabbitMQ 3.7.15、Redis 5.0、MongoDB 4.2.5、Mysql5.7等技术,采用Docker容器化部署。

mall学习教程 简介 mall学习教程,架构、业务、技术要点全方位解析。mall项目(40k+star)是一套电商系统,使用现阶段主流技术实现。涵盖了SpringBoot 2.3.0、MyBatis 3.4.6、Elasticsearch 7.6.2、RabbitMQ 3.7.15、Redis 5

macro 11.7k Jan 8, 2023
mall-swarm是一套微服务商城系统,采用了 Spring Cloud Hoxton & Alibaba、Spring Boot 2.3、Oauth2、MyBatis、Docker、Elasticsearch、Kubernetes等核心技术,同时提供了基于Vue的管理后台方便快速搭建系统。mall-swarm在电商业务的基础集成了注册中心、配置中心、监控中心、网关等系统功能。文档齐全,附带全套Spring Cloud教程。

mall-swarm 友情提示 快速体验项目:在线访问地址。 全套学习教程:《mall学习教程》。 Spring Cloud全套教程:《SpringCloud学习教程》。 专属学习路线:学习不走弯路,整理了套非常不错的《mall专属学习路线》。 项目交流:想要加群交流项目的朋友,可以加入mall项目

macro 9.7k Jan 3, 2023
【咕泡学院实战项目】-基于SpringBoot+Dubbo构建的电商平台-微服务架构、商城、电商、微服务、高并发、kafka、Elasticsearch

咕泡商城- 微服务架构实战 咕泡商城是咕泡学院 Java架构课程中,帮助学员对于技术更好落地的一个实战项目,项目基于springboot2.1.6.RELEASE+Dubbo2.7.3 来构建微服务。 业务模块划分,尽量贴合互联网公司的架构体系。所以,除了业务本身的复杂度不是很高之外,整体的架构基本

Mic 4.5k Dec 26, 2022
🦄 开源社区系统:基于 SpringBoot + MyBatis + MySQL + Redis + Kafka + Elasticsearch + Spring Security + ... 并提供详细的开发文档和配套教程。包含帖子、评论、私信、系统通知、点赞、关注、搜索、用户设置、数据统计等模块。

Echo — 开源社区系统 项目上线到服务器之后可能会出现各种各样的 BUG,比如 Elasticsearch 服务启动失败导致搜索模块不可用,但是在本地运行是完全没问题的,所以各位小伙伴可以放心下载部署。 ?? 项目简介 Echo 是一套前后端不分离的开源社区系统,基于目前主流 Java Web

小牛肉 434 Jan 7, 2023
Official Elasticsearch Java Client

Elasticsearch Java Client The official Java client for Elasticsearch. Note: this project is still a work in progress. This client is meant to replace

elastic 230 Jan 8, 2023
mall4cloud微服务商城,基于Spring Cloud、Nacos、Seata、Mysql、Redis、RocketMQ、canal、ElasticSearch、minio的B2B2C微服务商城系统,采用主流的互联网技术架构、全新的UI设计 B2B2C微服务商城|小程序微服务商城|

README 前言 本商城是基于Spring Cloud、Nacos、Seata、Mysql、Redis、RocketMQ、canal、ElasticSearch、minio的微服务B2B2C电商商城系统,采用主流的互联网技术架构、全新的UI设计、支持集群部署、服务注册和发现以及拥有完整的订单流程等

null 3k Jan 1, 2023
An example on how to build a configurable widget to trigger external searches along with Liferay search to Elasticsearch.

Liferay External Searches An example on how to build a configurable widget to trigger external searches along with Liferay search to Elasticsearch. Ge

Louis-Guillaume Durand 4 Oct 25, 2021
The IK Analysis plugin integrates Lucene IK analyzer into elasticsearch, support customized dictionary.

IK Analysis for Elasticsearch The IK Analysis plugin integrates Lucene IK analyzer (http://code.google.com/p/ik-analyzer/) into elasticsearch, support

Medcl 14.6k Jan 5, 2023
A compact and highly efficient workflow and Business Process Management (BPM) platform for developers, system admins and business users.

Flowable (V6) Maven Central: Docker Images: License: Homepage: https://www.flowable.org/ flowable / flowəb(ə)l / a compact and highly efficient workfl

Flowable 6k Jan 7, 2023
Bank Statement Analyzer Application that currently runs in terminal with the commands: javac Application.java java Application [file-name].csv GUI coming soon...

Bank Statement Analyzer Application that currently runs in terminal with the commands: javac Application.java java Application [file-name].csv GUI coming soon...

Hayden Hanson 0 May 21, 2022
A Spring Boot Camel boilerplate that aims to consume events from Apache Kafka, process it and send to a PostgreSQL database.

SPRING-BOOT CAMEL BOILERPLATE This is a Spring-Boot Camel Application model that you can use as a reference to study or even to use in your company. I

Bruno Delgado 45 Apr 4, 2022
ZeroTurnaround Process Killer

ZT Process Killer Continuous Integration Quick Overview The project was created in ZeroTurnaround to have a stable base functionality of stopping runn

ZeroTurnaround 114 Dec 29, 2022
ZeroTurnaround Process Executor

ZT Process Executor Continuous Integration Quick Overview The project was created to merge similar functionality of projects at ZeroTurnaround into a

ZeroTurnaround 817 Dec 29, 2022
Squadio-App is a Users-Accounts financial system. exposes Rest APIs with JWT authentication/Authorization process .

squadio-app Description Squadio-App is a Users-Accounts financial system. exposes Rest APIs with JWT authentication/Authorization process . How to Run

Bashar Othman 1 Jan 29, 2022
Deploys an agent to fix CVE-2021-44228 (Log4j RCE vulnerability) in a running JVM process

-- This repository has been archived -- Further development of this tool will continue at corretto/hotpatch-for-apache-log4j2. Thanks for sharing, com

Volker Simonis 108 Dec 23, 2021
Quarkus extension for launching an in-process Wiremock server

Quarkus extension for running Wiremock in DEVELOPMENT mode This extension is currently not published in any maven repository. If you want to test this

Quarkiverse Hub 5 Dec 30, 2022