TorMap visualises the location of Tor relays on a world map.

Overview

TorMap

This project visualizes current and past Tor relays on a world map. The backend regularly downloads descriptors from TorProject Archive and saves a processed version in a local database. Currently, the required archive part makes up 33 GB for the available consensus descriptor years (2007 - 2021). Processing of a descriptor type only starts after all descriptors of the same type have been downloaded and saved to disk. Processed data can instantly be fetched by the frontend to be displayed on the world map.

UI screenshot

Development

Requirements

Make sure you have at least 100 GB of free disk space, since the downloaded archive and local DB will take up a lot of space.

On most Unix systems you can use the installation script ./install. It will try to use your package manager to install missing requirements. Depending on your shell you run the script with ./install or bash ./install.

If you use Windows or the ./install script failed, please install these manually:

Troubleshooting:

  • Make sure JAVA_HOME points to a Java JDK version >= 11.

Run development servers

Make sure you have installed all requirements. The ./run script will start the backend and frontend servers in separate terminals/tabs. If you have tmux installed, a new tmux session named tormap will be started. Once you kill the terminal, the server is shutdown. The backend console will stay at an executing percentage below 100%. This is normal behaviour with Gradle & Spring Boot.

  • Linux: Type ./run or bash ./run
  • Windows: Type run.bat

The backend will be available at http://localhost:8080/ and frontend at http://localhost:3000/. If the script fails, or you prefer to run the servers manually:

  1. Go into backend directory and run commands
    • ./gradlew or on Windows gradlew.bat
    • ./gradlew bootRun or on Windows gradlew.bat bootRun
  2. Go into frontend directory and run commands
    • yarn
    • yarn start

In a fresh project without any preprocessed DB or pre-downloaded archive the backend will start to download an archive > 33 GB in size. Once the first 3 GB of consensus descriptors have been downloaded, they will start processing. Once 30 GB of server descriptors have been downloaded, it will take 1-2 days to complete processing. Any missing descriptors released by the TorProject will twice a day be automatically downloaded and processed.

Processing of descriptors does not necessarily happen in a chronological order, but one month of descriptors is always processed together. Different descriptor types are handled in parallel. While the backend is processing descriptors, the frontend will always be able to display finished data. Frontend features like family grouping or relay details will only be available, if the corresponding server descriptors have also been processed.

Backend

The backend uses a Spring Boot standalone webserver and is written in Kotlin.

CLI commands

Make sure you are in the backend directory. On Windows use gradlew.bat instead of ./gradlew for all following commands.

  • ./gradlew: installs required backend packages
  • ./gradlew build: creates build and runs Flyway database migrations
  • ./gradlew bootRun: creates build, runs it and listens on http://localhost:8080/
  • ./gradlew bootJar: creates build with a fat JAR which contains all dependencies and resources in build/libs/
  • ./gradlew bootBuildImage: creates build and a docker image (make sure local docker daemon is running)
  • ./gradlew flywayMigrate: Migrates the database
  • ./gradlew flywayClean: Drops all objects in the configured schemas
  • ./gradlew flywayInfo: Prints the details and status information about all the migrations
  • ./gradlew flywayValidate: Validates the applied migrations against the ones available on the classpath
  • ./gradlew flywayBaseline: Baselines an existing database, excluding all migrations up to and including baselineVersion
  • ./gradlew flywayRepair: Repairs the schema history table
  • ./gradlew dokkaHtml: generate code documentation in HTML format in build/dokka/

Config

The main backend config is located at backend/srv/main/resorces/application.properties. Logging options can be configured with backend/srv/main/resorces/logback-spring.xml. Dependencies are managed with Gradle and located at backend/build.gradle.kts.

OpenAPI specification

An interactive Swagger UI is available under http://localhost:8080/documentation and the specification can also be viewed in raw JSON under http://localhost:8080/documentation/json.

Database

TorMap uses an embedded H2 database which saves the whole state in a single DB file located at backend/resources/database/tormap.mv.db. If you want to use a DB already containing a few processed months, you can download one from https://lightningpuzzle.com/tormap/ and put it here backend/resources/database/tormap.mv.db.

To manually connect to the DB you either can add the datasource in your IDE or open http://localhost:8080/h2 while the backend is running. Make sure to configure the connection the same way your application.properties are set. In an IDE it might be necessary to configure the datasource URL with an absolute path to ensure the correct working directory is used.

Preprocessed databases:

IP to Autonomous Systems

TorMap uses previously imported Autonomous System (AS) data from IP2Location to get the AS of a Tor node's IP address. The mapping is done after a server descriptors file was processed and additionally every 12 hours (configurable). If you are not starting with a preprocessed TorMap DB you will need to import a CSV file containing autonomous systems into the local H2 database. It is advised to reimport a new CSV file every few months, to keep the IP ranges up to date.

  1. Create a free account at https://lite.ip2location.com/sign-up
  2. Download latest IPv4 CSV file from https://lite.ip2location.com/database-asn
  3. Run command ./gradlew build in backend directory
  4. Connect to a DB query console and run following commands:
    • TRUNCATE TABLE AUTONOMOUS_SYSTEM;
    • INSERT INTO AUTONOMOUS_SYSTEM SELECT * FROM CSVREAD('<absolute_path_to_csv_file>');

IP to geo location

TorMap uses a binary DB file from IP2Location to map IPv4 addresses of Tor nodes to geo locations. The mapping is applied when a consensus descriptor is being processed. It is advised to replace the binary file every few months, to keep the IP ranges up to date.

  1. Create a free account at https://lite.ip2location.com/sign-up
  2. Download latest IPv4 BIN file from https://lite.ip2location.com/database/db5-ip-country-region-city-latitude-longitude
  3. Replace old BIN file with new one in backend/resources/database/ip2location/IP2LOCATION-LITE-DB5.BIN

Frontend

The frontend uses a ReactJS web app together with TypeScript and Material-UI.

CLI commands

Make sure you are in the frontend directory. You can learn more about the following commands in the Create React App documentation.

  • yarn: installs required frontend packages
  • yarn start: creates build, runs it and listens by default on http://localhost:3000 (page reloads if you save frontend changes)
  • yarn build: creates production ready build in build folder

Config

The main frontend config is located at frontend/srv/util/config.ts. Further environment options like enable/disable Browser autostart and default port can be configured in frontend/.env. Dependencies are managed with yarn and located in frontend/package.json. Compiler options for TypeScript are located at frontend/tsconfig.json.

Build and run project

First make sure you have installed all requirements for development.

Backend

Build fat JAR:

  1. Go to backend directory where file gradlew is located
  2. Run command: ./gradlew && ./gradlew bootJar
  3. A fat jar containing all packages should now be located in backend/build/libs/.
  4. Create a folder containing a copy of the generated jar and the backend/resources folder.
  5. Go into the directory where the .jar file is located
  6. Run command java -jar <backend jar file>
  7. Backend should be available at http://localhost:8080

Build docker image:

  1. Go to backend directory where file gradlew is located
  2. Make sure docker is installed and the docker daemon is running
  3. Run command: ./gradlew && ./gradlew bootBuildImage
  4. A new image named juliushenke/tormap should be available in your local docker registry
  5. Run image in new container with command: docker run -p 8080:8080 juliushenke/tormap (optionally add the flag -v ~/tormap.mv.db:/workspace/resources/database/tormap.mv.db to mount a preprocessed DB from your host file system into the container)
  6. Backend should be available at http://localhost:8080

Frontend

  1. Go to frontend directory where file package.json is located
  2. Run command: yarn build
  3. A frontend/build folder should be generated containing all necessary frontend files
  4. Go into the directory where index.html is located
  5. Install serve and run command: serve -l 3000
  6. Frontend should be available at http://localhost:3000
Comments
Releases(0.0.2)
  • 0.0.2(Apr 12, 2022)

  • 0.0.1(Feb 17, 2022)

    Features

    Settings

    • group relays by family, country, coordinates and heatmap
    • show different relay types
    • filter relay flags
    • date picker and date slider

    Stats

    • 'Exit', 'Guard', 'Other relays' and total count
    • family count
    • country count

    Dialogs

    • family selection
    • relay selection
    • relay details
    • about TorMap
    Source code(tar.gz)
    Source code(zip)
Owner
TorMap
TorMap visualises the location of Tor relays on a world map.
TorMap
Realtime SOS Android Application. Location (GPS + Cellular Network) tracing application by alerting guardians of the User.

WomenSaftey Women Safety Android Application: Realtime SOS Android Application. Designed a Location (GPS + Cellular Network) tracing application by al

jatin kasera 6 Nov 19, 2022
Clone of real world Chatting application Whatsapp built on Android Studio and Firebase

WhatsappChatApp About This Project Clone of real world Chatting application Whatsapp built on Android Studio and Firebase Programming Language Used :

Aditya Bonde 11 May 23, 2022
Tree View; Mind map; Think map; tree map; custom view; 自定义; 树状图;思维导图;组织机构图;层次图

GysoTreeView 【中文】【English】 ⭐ If ok, give me a star ⭐ ⭐ ⭐ ⭐ ⭐ ⭐ Tree View; Mind map; Think map; tree map; 树状图;思维导图;组织机构图;层次图;树型图 A custom tree view for

怪兽N 303 Dec 30, 2022
Squaremap is a minimalistic and lightweight world map viewer for Minecraft servers, using the vanilla map rendering style

squaremap squaremap (formerly known as Pl3xMap) is a minimalistic and lightweight live world map viewer for Minecraft servers. What is squaremap If, l

Jason 204 Jan 3, 2023
null 5 Jan 11, 2022
BAIN Social is a Fully Decentralized Server/client system that utilizes Concepts pioneered by I2P, ToR, and PGP to create a system which bypasses singular hosts for data while keeping that data secure.

SYNOPSIS ---------------------------------------------------------------------------------------------------- Welcome to B.A.I.N - Barren's A.I. Natio

Barren A.I. Wolfsbane 14 Jan 11, 2022
Inspired by Freedom Wrapper, re-imagined with Tor

Inspired by The Freedom Wrapper The Freedom Wrapper Project is an Open Source and Free Android source code project. The project has moved to an organi

Hunter Burningham 4 Jun 29, 2022
Get device location by telephony (SIM card) or settings without using GPS tracker.

react-native-device-country Get device location by telephony (SIM card) or settings without using GPS tracker Installation yarn add react-native-devic

dev.family 46 Nov 29, 2022
Anthos Edge Use Cases for bringing apps and computation closer to the location where the action is, to improve response times and save bandwidth.

Anthos Bare Metal Edge Use Cases Edge computing is a distributed computing paradigm that brings computation and data storage closer to the location wh

Google Cloud Platform 27 Dec 20, 2022
Realtime SOS Android Application. Location (GPS + Cellular Network) tracing application by alerting guardians of the User.

WomenSaftey Women Safety Android Application: Realtime SOS Android Application. Designed a Location (GPS + Cellular Network) tracing application by al

jatin kasera 6 Nov 19, 2022
A library for off-line inverse geocoding that supports location up to district level

jetgeo 是一个用于离线逆地理编码的库,支持转换位置到县/地区级别, 通过它你可以消耗一定的内存以换取一个内存级别的转换。主要适用于精度要求不高的一些服务端场景。 如果你需要一个精确定位的场景,那么它可能并不合适,此时你可能需要适用地理位置信息提供商的一些服务, 但他们往往有次数限制或者需要支付

linG 6 Aug 25, 2022
Xposed module to hide the mock location setting

Hide Mock Location is an Xposed Module (now LSPosed on Android 11), which hides information about the 'Allow mock locations' setting.

null 27 Dec 15, 2022
Android library that allows you to determine your location in a few of lines!

locsimple Android library that allows you to determine your location in some lines! Benefits: automatic processing of permissions processing of enabli

Dmitry 7 Aug 31, 2022
Vector map library and writer - running on Android and Desktop.

Mapsforge See the integration guide and changelog. And read through how to contribute guidelines. If you have any questions or problems, don't hesitat

mapsforge 1k Dec 30, 2022
Java port of a concurrent trie hash map implementation from the Scala collections library

About This is a Java port of a concurrent trie hash map implementation from the Scala collections library. It is almost a line-by-line conversion from

null 147 Oct 31, 2022
A java.util.HashMap compatible map that won't stall puts or gets when resizing

/* * Written by Gil Tene, based on Apache Harmony version of java.util.HashMap. */ PauselessHashMap: A java.util.HashMap compatible Map implementat

Gil Tene 145 Oct 19, 2022
Vector map library and writer - running on Android and Desktop.

Mapsforge See the integration guide and changelog. And read through how to contribute guidelines. If you have any questions or problems, don't hesitat

mapsforge 1k Jan 7, 2023
🗺️ Minecraft map editor and mod

A Minecraft Map Editor... that runs in-game! With selections, schematics, copy and paste, brushes, and scripting! Use it in creative, survival in sing

EngineHub 2.7k Jan 1, 2023