Split into data blocks,In this format, efficient reading can be realized,Avoid unnecessary data reading operations.

Overview

image dataTear

  • 切换至:中文文档

  • knowledge base

  • dataTear

    Split into data fragments for data management. In this format, efficient reading can be achieved to avoid unnecessary data reading operations.

  • MAVEN dependent

    Maven repository url: https://s01.oss.sonatype.org/content/repositories/snapshots

    <repositories>
        <repository>
            <id>a</id>
            <name>sonatype</name>
            <url>https://s01.oss.sonatype.org/content/repositories/snapshots</url>
        </repository>
    </repositories>
    
    <dependencies>
        <dependency>
            <groupId>io.github.BeardedManZhao</groupId>
            <artifactId>dataTear</artifactId>
            <version>1.4-SNAPSHOT</version>
        </dependency>
    </dependencies>
    
  • Example of use

    The following is an example of reading and writing the dataTear file. The master is the data output component, and the Reader is the data reading component. The hyper-interfaces of these two components are the same, which is more flexible! The parameters can be set in chain mode. Of course, you can also set them step by step, providing strong flexibility. Please refer to the following main function code document for specific usage!

  • Full API Example

The API calls here are relatively complete, and the functions used are relatively comprehensive. You can use the following API calls for integrated development.

public static void main(String[] args) throws IOException {
        BasicConfigurator.configure();
        Date date = new Date();
        // Instantiate "Master" through RW, When instantiating, external data components can be integrated into this class, or the RW interface can be directly called to extract data components from the algorithm library
        DTMaster dtMaster = new DTMaster(s -> RW.getDT_UDF_Stream(DT_builtIn_UDF.LOCAL_GZIP).writeStream(s)) 
                .ReadFormat(DataSourceFormat.built_in).WriterFormat(DataOutputFormat.UDT) // Set data input and output mode
                .setUseSynchronization(true) // Whether to use synchronous data writing and wait for data output to complete
                .setIn_FilePath("D:\\InternetInformation.txt") // Set the read file path
                .setOUT_FilePath("C:\\Users\\4\\Desktop\\out") // Set the directory to which DataTear data is output
                .setSplitrex(",") // Set the column separator for data input
                .setOutSplit(",") // Set the column separator for data output
                .setPrimaryNum(0) // Set the primary key index in the data table. The data of the index column will be used as part of the nameManager
                .setFragmentationNum(2); // Set the number of output data fragments
        // Running components
        runRW(dtMaster);

        System.err.println("ok !  Time consuming to write data:" + (new Date().getTime() - date.getTime()) + "");

        /*TODO 数据组件分割 */

        Date date2 = new Date();
        // Instantiate "DTRead" through RW, When instantiating, external data components can be integrated into this class, or the RW interface can be directly called to extract data components from the algorithm library
        Reader dtRead = new DTRead(s -> RW.getDT_UDF_Stream(DT_builtIn_UDF.LOCAL_GZIP).readStream(s))
                .setPrimaryCharacteristic(data -> true) // Set the data primary key description. The data fragment of the primary key meeting this condition will be read
                .setUseMultithreading(true) // Set whether to use synchronous read
                .setMaxOutTimeMS(10000) // Set the maximum timeout (ms) for data reading. If the timeout is exceeded, data reading will be stopped immediately
                .setIn_FilePath("C:\\Users\\4\\Desktop\\out\\NameManager.NDT"); // Set the read NameManager path
        // Running components
        runRW(dtRead);

        System.err.println("ok !  Time consuming to read data:" + (new Date().getTime() - date2.getTime()) + "millisecond");
        System.err.println("source file:" + dtRead.getSrcFile() + "\tCreation time:" + new Date(dtRead.getCreateDateMS()).toLocaleString());
        System.err.println("Number of data rows:" + dtRead.getDataString().split("\n").length);
    }

    /**
     * 运行一个rw组件
     */
    public static boolean runRW(RW rw) throws IOException {
        return rw.openStream() && rw.op_Data() && rw.closeStream();
}
  • The simplest API example

If your customization requirements for functions are not so strong, this API call will be more suitable for you. It sets the necessary parameters. Note the data output mode here. If you need to call custom components or data components in the algorithm library, you need to change this mode to "UDT".

public static void main(String[] args) throws IOException {
    BasicConfigurator.configure();
    // Configure Data Output Class
    DTMaster dtMaster = new DTMaster(null)
            .WriterFormat(DataOutputFormat.built_in) // be careful!!! If UDF is not set here, data will be automatically written in "LOCAL_TEXT" mode
            .setPrimaryNum(0)
            .setIn_FilePath("C:\\Users\\4\\Desktop\\mathematicalModeling\\Attached documents\\test.txt") // Set the path of the converted file
            .setOUT_FilePath("C:\\Users\\4\\Desktop\\mathematicalModeling\\out") // Set the storage path of NM and other files after conversion
            .setSplitrex("\\s+");
    dtMaster.openStream();
    dtMaster.op_Data();
    dtMaster.closeStream();

    // Configure Data Reading Class
    Reader reader = new DTRead(InPath -> RW.getDT_UDF_Stream(DT_builtIn_UDF.LOCAL_TEXT).readStream(InPath))
            .setPrimaryCharacteristic((data) -> true)
            .setIn_FilePath("C:\\Users\\4\\Desktop\\mathematicalModeling\\out\\NameManager.NDT"); // Set the NM path of the read file
    reader.openStream();
    reader.op_Data();
    reader.closeStream();

    System.out.println(reader.getDataString());
}
  • DTMaster component output mode

image

  • built_in

The mode of using the built-in data output component is equivalent to the data output component of "LOCAL_TEXT". In this mode, you do not need to load the data output component. However, the disadvantage is that when using the built-in data component output mode, you can only use "LOCAL_TEXT" for data output. The setting method is shown in the following figure.

image

  • UDF

UDF is also a user-defined data output mode. In this mode, you must load a data output component when instantiating the DTMaster. The loading of this data component can be extracted from the algorithm library or directly implemented by yourself. The following is an example of calling the local GZIP data output component in the algorithm library. image

directory structure

KnowledgeDocument

The knowledge base file archive is used for the storage task of DataTear series knowledge documents. You can access it directly through the documents on the home page. There is no need to enter here directly. It is messy internally.

src_code

The source code storage directory of DataTear, where you can view the relevant source code of DataTear. Of course, here is the latest source code, which you can use to compile, so that you can obtain the latest version.

Note: The latest version is often unstable, so we recommend you to use the version that has been released for a long time!

README-Chinese.md

The Chinese version of the DaraTear homepage document. You can switch languages on the homepage default page to access this file.

README.md

The default version of the DaraTear homepage document. You can directly access this file on the homepage!

You might also like...

By this package we can get sim info, call logs and sms logs.Also we can find for specific sim info and call logs as well.

sim_sms_call_info A new flutter plugin project. Getting Started This project is a starting point for a Flutter plug-in package, a specialized package

Sep 17, 2022

A small library for parsing ItemStacks from a human-readable format

easy-item A small library for parsing ItemStacks from a human-readable format (1.16.5+, Java 11) TODO: Maybe add serialization (item to human-readable

Dec 4, 2021

Spring-Boot-Plus is a easy-to-use, high-speed, high-efficient,feature-rich, open source spring boot scaffolding

Spring-Boot-Plus is a easy-to-use, high-speed, high-efficient,feature-rich, open source spring boot scaffolding

Everyone can develop projects independently, quickly and efficiently! What is spring-boot-plus? A easy-to-use, high-speed, high-efficient, feature-ric

Dec 31, 2022

A compact and highly efficient workflow and Business Process Management (BPM) platform for developers, system admins and business users.

Flowable (V6) Maven Central: Docker Images: License: Homepage: https://www.flowable.org/ flowable / flowəb(ə)l / a compact and highly efficient workfl

Jan 7, 2023

A simple and efficient short URL conversion service based on SpringBoot.

A simple and efficient short URL conversion service based on SpringBoot.

A simple and efficient short URL conversion service based on SpringBoot What is GeniusShortUrl? GeniusShortUrl is an open source reactive service that

Apr 8, 2022

Efficient yet Effective plugin to catch X-rayers.

Efficient yet Effective plugin to catch X-rayers.

XCatch Efficient yet effective plugin to catch X-rayers for Minecraft 1.13-1.18. XCatch analizes how a player is mining by looking it how many turns i

Nov 10, 2022

Photo live wallpaper with auto dark mode and power-efficient animations

Photo live wallpaper with auto dark mode and power-efficient animations

Pallax Android: Photo Live Wallpaper Pallax Android is an Android app that lets you convert your current static home screen background into a stunning

Dec 17, 2022

The Apache Commons CSV library provides a simple interface for reading and writing CSV files of various types.

Apache Commons CSV The Apache Commons CSV library provides a simple interface for reading and writing CSV files of various types. Documentation More i

Dec 26, 2022

Application to benchmark block reading from bitcoind

BlockReader BlockReader is a small command line application to benchmark block reading performance. Currently, it is using bitcoin-cli to read blocks

Jan 18, 2022
Comments
  • Bump commons-io from 2.6 to 2.7 in /src_code1.4

    Bump commons-io from 2.6 to 2.7 in /src_code1.4

    Bumps commons-io from 2.6 to 2.7.

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Releases(v1.4.1)
  • v1.4.1(Sep 28, 2022)

    • introduce Thank you for using. This is my packed jar,You can directly import the jar into the project!
    • Contact me phone: +86 17802240898 postbox:[email protected]
    • Update content Optimize the Maven dependency structure, and upgrade the Maven dependency version to avoid hidden vulnerabilities to the greatest extent.
    • Use jar The jar contains the most basic operation requirements, which can simplify the dependent configuration. If you have the requirement to rely on customization, please refer to Maven Dependent Import. 3feeef21f8e32d0ab1f72452a561a99
    • Maven Dependent Import This is the dependency configuration of the framework pulled from the central repository. If you have imported the jar package, you do not need to configure the following maven dependencies.
    <repositories>
        <repository>
            <id>a</id>
            <name>sonatype</name>
            <url>https://s01.oss.sonatype.org/content/repositories/snapshots</url>
        </repository>
    </repositories>
    
    <dependencies>
        <dependency>
            <groupId>io.github.BeardedManZhao</groupId>
            <artifactId>dataTear</artifactId>
            <version>1.4.1-SNAPSHOT</version>
        </dependency>
    </dependencies>
    
    Source code(tar.gz)
    Source code(zip)
    DataTear-1.4.1.jar(4.74 MB)
  • v1.4(Sep 21, 2022)

Owner
LingYuZhao
LingYuZhao
1.7 - 1.18 utility class that allows you to change blocks at blazing fast speeds

BlockChanger 1.7 - 1.18 utility class that allows you to change blocks at blazing fast speeds Setup Just import the class to your project. Usage Playe

null 25 Dec 23, 2022
Extremely simple and easy to use lucky blocks plugin!

SimpleLuckyBlocks plugin, the new fully customisable lucky blocks plugin Ever felt the need for a free, yet simple lucky blocks plugin? Well you've fo

Steven Zhu 4 Jun 8, 2022
UMS is a CRUD based management system which uses File Handling to manipulate data and perform the CRUD operations

UMS is a CRUD (Create, Read, Update, Delete) based management system which uses File Handling to manipulate data and perform the CRUD operations. It is a group project made using Java procedural programming having both User and Admin sides.

Daoud-Hussain 9 Dec 20, 2022
Basic crud operations with json data, main focus is with tests

Spring Crud operations Basic crud operations with json data, main focus is with tests. For future reference Road Map Basic Crud on controllers (done)

Jarno Saastamoinen 1 Feb 1, 2022
It creates a Trie with given input and perform character based operations

Trie-with-character-based-operations It creates a Trie with given input and perform character based operations Boolean Search(String arg): This functi

null 2 Jul 3, 2022
An intelliJ plugin providing a UI layer for git-flow, which in itself is a collection of Git extensions to provide high-level repository operations for Vincent Driessen's branching model.

Git Flow Integration Plus for Intellij An intelliJ plugin providing a UI layer for git-flow, which in itself is a collection of Git extensions to prov

RubinCarter 35 Nov 8, 2022
It contains a simple program to apply basic arithmetic operations in Morse code

Morse-Calculator By Mohamad Farag F. Makkawi This project falls under DSL (Domain Specific Language) . the Input is an equation in which simple arithm

Mohamad Makkawi 1 Apr 29, 2022
CO1212 - Stack Operations Activity

StackOperations Download this two java files and implement the methods which are mentioned inside the class. Just use the basic things that I have tau

Dilshan Karunarathne 4 Apr 26, 2022
Teaching repository for the undergraduate course in Operations Research at Technical University Munich.

Tutorial for Operations Research SS22 Konstantin Kuchenmeister Teaching repository for the undergraduate course in Operations Research at Technical Un

Konstantin Kuchenmeister 9 Aug 27, 2022
Discourse-java is a platform where users can freely discuss on topics they want to, and like-minded people can join in and contribute

Discourse is the 100% open source discussion platform built for the next decade of the Internet. Use it as a: mailing list discussion forum long-form

Infosys Ltd 16 Oct 19, 2022