Data extraction from smartphones and GPS and Accelerometer data "fusion" with Kalman filter.

Overview

This is library for GPS and Accelerometer data "fusion" with Kalman filter. All code is written in Java. It helps to increase position accuracy and GPS distance calculation on Android devices for the driver's and couriers' apps. And also, it may be used for precise tracking in on-demand services.

Project consists of 2 parts:

  • Main: GpsAccelerationKalmanFusion module;
  • 2 helper applications:

License: MIT

What can "Smartphone Bot Localization system" do?

This module helps to increase GPS coordinates accuracy and also:

  • reduces the errors in route tracking;
  • decreases the noise from Low-class smartphones;
  • excludes sharp «jumps» to the points remote from a real route;
  • eliminates additional distance when the object is motionless;
  • filters errors duу to the short-term loss of GPS-signal.

How to use

There is example application in current repository called "Sensor Data Collector".

WARNING!!

Right now these sensors should be available:
TYPE_ROTATION_VECTOR, TYPE_LINEAR_ACCELERATION.

It's possible to use just TYPE_ACCELEROMETER with high-pass filter.
Also it's possible to use Madgwick filter instead of rotation vector, but gyroscope and magnetometer sensors should be available in that case.

KalmanLocationService

This is main class. It implements data collecting and processing. You need to make several preparation steps for using it:

  1. Add to application manifest this:
<service
            android:name="mad.location.manager.lib.Services.KalmanLocationService"
            android:enabled="true"
            android:exported="false"
            android:stopWithTask="false" />
  1. Create some class and implement LocationServiceInterface and optionally LocationServiceStatusInterface .
  2. Register this class with ServicesHelper.addLocationServiceInterface(this) (do it in constructor for example)
  3. Handle locationChanged callback. There is Kalman filtered location WITHOUT geohash filtering. Example of geohash filtering is in MapPresenter class.
  4. Init location service settings object (or use standard one) and pass it to reset() function.

Important things!

It's recommended to use start(), stop() and reset() methods, because this service has internal state. Use start() method at the beginning of new route. Stop service when your application doesn't use locations data. That need to be done for decrease battery consumption.

Kalman filter settings

There are several settings for KalmanFilter. All of them stored in KalmanLocationService.Settings class.

  • Acceleration deviation - this value controls process noise covariance matrix. In other words it's "trust level" of accelerometer data. Low value means that accelerometer data is more preferable.
  • Gps min time - minimum time interval between location updates, in milliseconds
  • Gps min distance - minimum distance between location updates, in meters
  • Sensor frequency in Herz - the rate sensor events are delivered at
  • GeoHash precision - length of geohash string (and precision)
  • GeoHash min point - count of points with same geohash. GPS point becomes valid only when count greater then this value.
  • Logger - if you need to log something to file just implement ILogger interface and initialize settings with that object. If you don't need that - just pass null .

There is an example in MainActivity class how to use logger and settings.

GeoHashRTFilter

There are 2 ways of using GeoHash real-time filter :

  • It could be used as part of KalmanLocationService. It will work inside that thread and will be used by service. But then you need to use start(), stop() and reset() methods.
  • It could be used as external component and filter Location objects from any source (not only from KalmanLocationService). You need to reset it before using and then use method filter() .

It will calculate distance in 2 ways : Vincenty and haversine formula . Both of them show good results so maybe we will add some flag for choose.

The roadmap

Visualizer

  • Implement some route visualizer for desktop application
  • Implement Kalman filter and test all settings
  • Implement noise generator for logged data
  • Improve UI. Need to use some controls for coefficient/noise changes

Filter

  • Implement GeoHash function
  • Get device orientation
    • Get device orientation using magnetometer and accelerometer + android sensor manager
    • Get device orientation using magnetometer, accelerometer and gyroscope + Madgwick AHRS
    • Get device orientation using rotation vector virtual sensor
  • Compare result device orientation and choose most stable one
  • Get linear acceleration of device (acceleration without gravity force)
  • Convert relative linear acceleration axis to absolute coordinate system (east/north/up)
  • Implement Kalman filter core
  • Implement Kalman filter for accelerometer and gps data "fusion"
  • Logger for pure GPS data, acceleration data and filtered GPS data.
  • Restore route if gps connection is lost

Library

  • Separate test android application and library
  • Add library to some public repository

Theory

Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, containing statistical noise and other inaccuracies, and produces estimates of unknown variables that tend to be more accurate than those based on a single measurement alone, by estimating a joint probability distribution over the variables for each timeframe.

You can get more details about the filter here.

The filter is a de-facto standard solution in navigation systems. The project simply defines the given data and implements some math.

The project uses 2 data sources: GPS and accelerometer. GPS coordinates are not very accurate, but each of them doesn't depend on previous values. So, there is no accumulation error in this case. On the other hand, the accelerometer has very accurate readings, but it accumulates error related to noise and integration error. Therefore, it is necessary to "fuse" these two sources. Kalman is the best solution here.

So first - we need to define matrices and do some math with them. And second - we need to get real acceleration (not in device orientation).

First one is described in current project's wiki. But second one is little bit more complex thing called "sensor fusion". There is a lot information about this in internet.

Algorithms

Sensor fusion is a term that covers a number of methods and algorithms, including:

For real acceleration we need to know 2 things: "linear acceleration" and device orientation. Linear acceleration is acceleration along each device axis excluding force of gravity. It could be calculated by high pass filter or with more complex algorithms. Device orientation could be calculated in many ways:

  • Using accelerometer + magnetometer
  • Using accelerometer + magnetometer + gyroscope
  • Using Madgwick filter
  • Using virtual "rotation vector" sensor.

Best results show Madgwick filter and ROTATION_VECTOR sensor, but Madgwick filter should be used when we know sensor frequency. Android doesn't provide such information. We can set minimum frequency, but it could be much higher then specified. Also we need to provide gain coefficient for each device. So best solution here is to use virtual ROTATION_VECTOR sensor. You can get more details from current project's wiki.

Issues

Feel free to send pull requests. Also feel free to create issues.

License

MIT License

Copyright (c) 2020 Mad Devs

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

You might also like...

Dynamically filters JPA entities with a simple query syntax. Provides JPA/Hibernate predicates and Spring Data specifications.

Spring Filter You need a way to dynamically filter entities without any effort? Just add me to your pom.xml. Your API will gain a full featured search

Nov 24, 2022

Realtime Data Processing and Search Engine Implementation.

Realtime Data Processing and Search Engine Implementation.

Mutad The name Mutad is a reverse spelling of datum. Overview An implementation of a real-time data platform/search engine based on various technology

Aug 4, 2022

Excel utility for Java to read and write data in declarative way.

Excel utility for Java to read and write data in declarative way.

Data Excel Exporter A Java wrapper using Apache POI to read and write Excel file in declarative fashion. Installation ExcelUtil is using Apache POI ve

Oct 16, 2022

Decorating Spring Boot Reactive WebClient for tracing the request and response data for http calls.

Decorating Spring Boot Reactive WebClient for tracing the request and response data for http calls.

SpringBoot Reactive WebClient 🔍 Tracing HTTP Request through a single pane of glass Decorating Spring Boot Reactive WebClient for tracing the request

Jul 13, 2022

Spring Data Redis extensions for better search, documents models, and more

Object Mapping (and more) for Redis! Redis OM Spring extends Spring Data Redis to take full advantage of the power of Redis. Project Stage Snapshot Is

Dec 6, 2022

Spring Boot & MongoDB Login and Registration example with JWT, Spring Security, Spring Data MongoDB

Spring Boot & MongoDB Login and Registration example with JWT, Spring Security, Spring Data MongoDB

Spring Boot Login and Registration example with MongoDB Build a Spring Boot Auth with HttpOnly Cookie, JWT, Spring Security and Spring Data MongoDB. Y

Dec 6, 2022

High Performance data structures and utility methods for Java

High Performance data structures and utility methods for Java

Agrona Agrona provides a library of data structures and utility methods that are a common need when building high-performance applications in Java. Ma

Dec 3, 2022

UMS is a CRUD based management system which uses File Handling to manipulate data and perform the CRUD operations

UMS is a CRUD based management system which uses File Handling to manipulate data and perform the CRUD operations

UMS is a CRUD (Create, Read, Update, Delete) based management system which uses File Handling to manipulate data and perform the CRUD operations. It is a group project made using Java procedural programming having both User and Admin sides.

Nov 17, 2022

Reads data from sentences from hundreds of movie reviews, and evaluates the sentiment level of sentences entered by the user.

Sentiment_Analyzer Reads data from sentences from hundreds of movie reviews included in reviews.txt. Assigns a value to each word depending on the rev

Jun 13, 2022
Owner
Rahul Goel
Former SWE Intern @ Microsoft | Chief Engineer @ Team AVERERA | System Engineer @ Drobot Incc. | IIT(BHU) Varanasi
Rahul Goel
Get device location by telephony (SIM card) or settings without using GPS tracker.

react-native-device-country Get device location by telephony (SIM card) or settings without using GPS tracker Installation yarn add react-native-devic

dev.family 46 Nov 29, 2022
A proof-of-concept Android application to detect and defeat some of the Cellebrite UFED forensic toolkit extraction techniques.

LockUp An Android-based Cellebrite UFED self-defense application LockUp is an Android application that will monitor the device for signs for attempts

levlesec 301 Nov 30, 2022
Representational State Transfer + Structured Query Language(RSQL): Demo application using RSQL parser to filter records based on provided condition(s)

Representational State Transfer + Structured Query Language: RSQL Demo application using RSQL parser to filter records based on provided condition(s)

Hardik Singh Behl 9 Nov 23, 2022
In this project, we will implement two Spring Boot Java Web application called, streamer-data-jpa and streamer-data-r2dbc.

In this project, we will implement two Spring Boot Java Web application called, streamer-data-jpa and streamer-data-r2dbc. They both will fetch 1 million of customer's data from MySQL and stream them to Kafka. The main goal is to compare the application's performance and resource utilization.

Ivan Franchin 6 Nov 2, 2022
Repositório referente ao código de uma classe data, com testes JUNIT, classe de exceção própria e classe aplicação para demonstrar as diversas funcionalidades da classe data

Exercicio-Data Repositório referente ao código de uma classe data, com testes JUNIT, classe de exceção própria e classe aplicação para demonstrar as d

Bruno Silveira Cequeira Lima 3 May 4, 2021
Let Litematica be able to paste tile entity data of block / entity data in a server

Litematica Server Paster Let Litematica be able to paste tile entity data of block / entity data in a server By using a custom chat packet to bypass t

Fallen_Breath 21 Nov 28, 2022
Split into data blocks,In this format, efficient reading can be realized,Avoid unnecessary data reading operations.

dataTear 切换至:中文文档 knowledge base dataTear Split into data fragments for data management. In this format, efficient reading can be achieved to avoid un

LingYuZhao 25 Nov 30, 2022
An Open-Source repository 🌎 that contains all the Data Structures and Algorithms concepts and their implementation, programming questions and Interview questions

An Open-Source repository ?? that contains all the Data Structures and Algorithms concepts and their implementation, programming questions and Interview questions. The main aim of this repository is to help students who are learning Data Structures and Algorithms or preparing for an interview.

Aritra Das 17 Oct 23, 2022
Cosmic Ink is a transcript application which was built with the help of Symbl AI and At Sign platform for back-end to store our data and authenticate

Cosmic-Ink Cosmic Ink is a transcript application which was built with the help of Symbl AI and At Sign platform for back-end to store our data and au

Venu Sai Madisetti 4 Dec 1, 2022
SecureDB is an extension for Ai2 Appinventor and its distros which stores the data in the form of key and value just like TinyDB but in a more secure manner.

SecureDB SecureDB is an extension for Ai2 Appinventor and its distros which stores data for your app in a secure format locally on user's device. Expl

Akshat Developer 3 Sep 24, 2022