Realtime Data Processing and Search Engine Implementation.

Related tags

Spring Boot java docker aws elasticsearch cloud typescript kibana kafka spring-boot storm microservice reactjs nextjs material-ui data-stream-processing data-stream-mining
Overview

Mutad

The name Mutad is a reverse spelling of datum.

Overview

An implementation of a real-time data platform/search engine based on various technology stack. The implementation is rather for my own learning purpose.

The above gif image demonstrates a real-time visualization of the tweet geoparsing of Mutad. According to my proceeding research, the amount of tweets with geospatial metadata is quite a few, say 10 per 10,000 tweets, from the Hosebird data feed. Thus I took another strategy to extract geospatial data from the tweet. Mutad implements real-time geoparsing based on CLAVIN. Because of my poor implementation, the accuracy is not that high yet.

The above gif image demonstrates a list timeline displaying the latest Tweets ordered from newest to oldest with a specified search query.

Architecture

This diagram illustrates the overall architecture of Mutad. The technology stack is as follows:

Getting Started

Launch services by following the instructions below:

Future Work

The project is still WIP. See Issues.

You might also like...

DatasetCreator is a lightweight RESTFul client implementation of the Salesforce CRM Analytics External Data API.

DatasetCreator is a lightweight RESTFul client implementation of the Salesforce CRM Analytics External Data API. It has been deliberately developed with no 3rd party jars with the goal of being a lean, reliable and scalable solution.

Dec 16, 2022

Data extraction from smartphones and GPS and Accelerometer data "fusion" with Kalman filter.

Data extraction from smartphones and GPS and Accelerometer data

This is library for GPS and Accelerometer data "fusion" with Kalman filter. All code is written in Java. It helps to increase position accuracy and GP

Nov 22, 2022

This is a simple realization of custom messages pre/post processing in spring-boot HTTP/Stream requests & responses

spring-boot-custom-message-converting-instances This is a simple realization of custom messages converting in spring-boot HTTP requests and responses.

Jul 22, 2022

PipelinR is a lightweight command processing pipeline ❍ ⇢ ❍ ⇢ ❍ for your Java awesome app.

PipelinR PipelinR is a lightweight command processing pipeline ❍ ⇢ ❍ ⇢ ❍ for your awesome Java app. PipelinR has been battle-proven on production, as

Jan 8, 2023

Genting SkyWorlds Theme Park’s Entrance Ticket Processing

Genting SkyWorlds Theme Park’s Entrance Ticket Processing A program to book normal or express ticket(s) (Express tickets people does not have to wait

Aug 6, 2022

In this project, we will implement two Spring Boot Java Web application called, streamer-data-jpa and streamer-data-r2dbc.

In this project, we will implement two Spring Boot Java Web application called, streamer-data-jpa and streamer-data-r2dbc.

In this project, we will implement two Spring Boot Java Web application called, streamer-data-jpa and streamer-data-r2dbc. They both will fetch 1 million of customer's data from MySQL and stream them to Kafka. The main goal is to compare the application's performance and resource utilization.

Nov 2, 2022

Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.

Cadence This repo contains the source code of the Cadence server and other tooling including CLI, schema tools, bench and canary. You can implement yo

Jan 4, 2023

Repositório referente ao código de uma classe data, com testes JUNIT, classe de exceção própria e classe aplicação para demonstrar as diversas funcionalidades da classe data

Exercicio-Data Repositório referente ao código de uma classe data, com testes JUNIT, classe de exceção própria e classe aplicação para demonstrar as d

May 4, 2021

Let Litematica be able to paste tile entity data of block / entity data in a server

Litematica Server Paster Let Litematica be able to paste tile entity data of block / entity data in a server By using a custom chat packet to bypass t

Dec 24, 2022
Comments
Releases(1.0.0)

  • 1.0.0(Aug 15, 2021)

    Version 1.0.0 of Mutad offers initial functionalities processing the Twitter Hosebird data stream. This list details most of the major changes made in Mutad v1.0.0:

    • Implement Tweet Timeline
    • Implement Tweet Geoparging
    • Implement Tweet Trend
    Source code(tar.gz)
    Source code(zip)
Owner
Shingo OKAWA
A Data Misfit.
Shingo OKAWA
GitHub
Search API with spelling correction using ngram-index algorithm: implementation using Java Spring-boot and MySQL ngram full text search index

Search API to handle Spelling-Corrections Based on N-gram index algorithm: using MySQL Ngram Full-Text Parser Sample Screen-Recording Screen.Recording

Hardik Singh Behl 5 Dec 4, 2021
VFX processing engine in Java

sofvfx Video converter for ASCII display. Setup To use sofvfx, you will need to download ffmpeg. When done, open the downloaded .zip, go to /bin/ and

angelsflyinhell 4 Dec 29, 2022
Spring Data Redis extensions for better search, documents models, and more

Object Mapping (and more) for Redis! Redis OM Spring extends Spring Data Redis to take full advantage of the power of Redis. Project Stage Snapshot Is

Redis 303 Dec 29, 2022
An Open-Source repository 🌎 that contains all the Data Structures and Algorithms concepts and their implementation, programming questions and Interview questions

An Open-Source repository ?? that contains all the Data Structures and Algorithms concepts and their implementation, programming questions and Interview questions. The main aim of this repository is to help students who are learning Data Structures and Algorithms or preparing for an interview.

Aritra Das 19 Dec 29, 2022
A blog recipes API for you to share and search for food recipes

A blog recipes API for you to share and search for food recipes

java dojo 3 Apr 15, 2022
Processing and node.js project for rendering MouseGAN images from RunwayML

Computer MouseGAN Processing and node.js project for rendering MouseGAN images from RunwayML. Usage Clone or download this repository. git clone https

Coding Train 17 Apr 1, 2022
An example on how to build a configurable widget to trigger external searches along with Liferay search to Elasticsearch.

Liferay External Searches An example on how to build a configurable widget to trigger external searches along with Liferay search to Elasticsearch. Ge

Louis-Guillaume Durand 4 Oct 25, 2021
A simple quarkus app with hibernate as ORM, used to show the features of Hibernate Search

beer-manager-hibernate-search Project This project uses Quarkus, the Supersonic Subatomic Java Framework. If you want to learn more about Quarkus, ple

Sedona Solutions 3 Jan 6, 2022
Platform for teachers & students, to offer or search for tutoring.

tutoring-platform Table of contents About The Project Technologies Getting Started Roadmap Contributing License About The Project Platform for teacher

null 10 Dec 27, 2022
A near-real-time Mesh Join Algorithm Implementation provided with a Complete Data warehouse for METRO

Mesh Join Algorithm and Data Warehouse A complete Mesh-Join Algorithm Implementation as provided in the paper R-MESHJOIN . This is demonstrated by the

null 3 Aug 11, 2022