Core part of pipes framework plus some commonly used extensions

Related tags

Miscellaneous pipes
Overview

Pipes

Pipes is a simple, lightweight data processing framework for Java. This repo comes with the core part plus three extensions (For Google Big Query, Google cloud storage and Amazon S3 storage). For more information please read the tutorial.

This project is licensed under the terms of the MIT license.

Installation

Pipes library jars are released via Maven Central Repository. The current version is 0.9.

Core artifact:

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-core</artifactId>
  <version>0.9</version>
</dependency>

Optional extensions:

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-google-cs</artifactId>
  <version>0.9</version>
</dependency>

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-google-bq</artifactId>
  <version>0.9</version>
</dependency>

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-amazon-s3</artifactId>
  <version>0.9</version>
</dependency>
You might also like...

Published on Maven Central and jCenter Java Library that compares 2 images with the same sizes and shows the differences visually by drawing rectangles. Some parts of the image can be excluded from the comparison. Can be used for automation qa tests.

Published on Maven Central and jCenter Java Library that compares 2 images with the same sizes and shows the differences visually by drawing rectangles. Some parts of the image can be excluded from the comparison. Can be used for automation qa tests.

About Configurations Release Notes Usage Demo License Contributing Code of Conduct Other Projects About Published on Maven Central and jCenter Java Li

Jan 9, 2023

Published on Maven Central and jCenter Java Library that compares 2 images with the same sizes and shows the differences visually by drawing rectangles. Some parts of the image can be excluded from the comparison. Can be used for automation qa tests.

Published on Maven Central and jCenter Java Library that compares 2 images with the same sizes and shows the differences visually by drawing rectangles. Some parts of the image can be excluded from the comparison. Can be used for automation qa tests.

About Configurations Release Notes Usage Demo License Contributing Code of Conduct Other Projects About Published on Maven Central and jCenter Java Li

Dec 21, 2022

Linked List - a part of the Collection framework present in java.util package

Linked List - a part of the Collection framework present in java.util package

Linked List is a part of the Collection framework present in java.util package. This class is an implementation of the LinkedList data structure which is a linear data structure where the elements are not stored in contiguous locations and every element is a separate object with a data part and address part

Mar 6, 2022

Twitter Text Libraries. This code is used at Twitter to tokenize and parse text to meet the expectations for what can be used on the platform.

twitter-text This repository is a collection of libraries and conformance tests to standardize parsing of Tweet text. It synchronizes development, tes

Jan 8, 2023

Facsimile - Copy Your Most Used Text to Clipboard Easily with Facsimile!. It Helps You to Store You Most Used Text as a Key, Value Pair and Copy it to Clipboard with a Shortcut.

Facsimile - Copy Your Most Used Text to Clipboard Easily with Facsimile!. It Helps You to Store You Most Used Text as a Key, Value Pair and Copy it to Clipboard with a Shortcut.

Facsimile An exact copy of Your Information ! Report Bug · Request Feature Table of Contents About The Project Built With Getting Started Installation

Sep 12, 2022

The core module is for the TenIO framework.

TenIO Core Module TenIO is an open-source project to create multiplayer online games that includes a java NIO (Non-blocking I/O) based server specific

Oct 23, 2022

CMU ARK Twitter Part-of-Speech Tagger

CMU ARK Twitter Part-of-Speech Tagger v0.3.2 http://www.ark.cs.cmu.edu/TweetNLP/ Basic usage for released version ================================ R

Dec 3, 2022

The Ludii general game system, developed as part of the ERC-funded Digital Ludeme Project.

The Ludii general game system, developed as part of the ERC-funded Digital Ludeme Project.

The Ludii General Game System Ludii is a general game system being developed as part of the ERC-funded Digital Ludeme Project (DLP). This repository h

Dec 25, 2022

A repository that contains the backend part of the Human Resources Management System.

Human Resources Management System Backend A human resources management system is a form of human resources (HR) software that combines several systems

Dec 26, 2022

This repo is created to help people with the machine coding interview. There is no free website to provide complete guide for machine coding round so I have created this repo where I have shared all my machine coding practices and created a medium post as well to help with theory part.

machineCoding This repo is created to help people with the machine coding interview. There is no free website to provide complete guide for machine co

Nov 3, 2022

This project was done as a part of Hug61B taught by Josh Hug.

This project was done as a part of Hug61B taught by Josh Hug.

byow This project was done as a final project for Hug61B taught by UC Berkeley's Josh Hug, The main aim of this project was to design a world generati

Sep 2, 2021

Share the chat messages across Minecraft Servers via HTTP backend powered by Spring Boot, this is the backend part of the project.

InterconnectedChat-Backend Share the chat messages across Minecraft Servers via HTTP backend powered by Spring Boot, this is the backend part of the p

Oct 6, 2021

This repository is for Todo application. This contains the Backend part of the application.

Todo Application 개요(Abstract) 개인용 할일 목록 리스트 앱플리케이션 구축 (Personal Todo List Application) 목적 1. React.js기초, AWS서버 활용, 스프링 부트 공부 목적으로 프로젝트 시작했습니다.

Jan 8, 2022

A version of the Arkanoid game, as part of the OOP course

A version of the Arkanoid game, as part of the OOP course

Arkanoid! About Dependencies Installation instructions About I implemented a version of the Arkanoid game, as part of the OOP course i took during my

Nov 8, 2022

Checks items in inventories to see if they were duped as part of the Crystal Hollows Dupe Event in Hypixel Skyblock.

Checks items in inventories to see if they were duped as part of the Crystal Hollows Dupe Event in Hypixel Skyblock.

Checks items in inventories to see if they were duped as part of the Crystal Hollows Dupe Event in Hypixel Skyblock.

Dec 19, 2022

Reimplementation of TheFlow's bd-jb. No kernel part yet.

BD-JB reimplementation based on TheFlow's report and presentation. Implements loading arbitrary .bin payloads using vulnerabilities #2 (privileged con

Dec 18, 2022

A part of the course Mobile Application Development

A part of the course Mobile Application Development

Movie-Recommendation-Android-App A part of the course Mobile Application Development Pre-processed TMDB dataset / TMDB API responses and built a conte

Dec 24, 2022

Restaurant Advisor application created for Munich as part of team project at the university.

Restaurant Advisor application created for Munich as part of team project at the university.

Restaurant Advisor Application It is important to mention that this application was created for educational purposes. Restaurants information is taken

Oct 16, 2022

基于代码生成器的低代码平台，超越传统商业平台！前后端分离架构SpringBoot 2.x，SpringCloud，Ant Design&Vue，Mybatis-plus，Shiro，JWT。强大的代码生成器让前后端代码一键生成，无需写任何代码! 引领新低代码开发模式OnlineCoding-代码生成-手工MERGE，帮助Java项目解决70%重复工作，让开发更关注业务，既能快速提高开发效率，帮助公司节省成本，同时又不失灵活性。

基于代码生成器的低代码平台，超越传统商业平台！前后端分离架构SpringBoot 2.x，SpringCloud，Ant Design&Vue，Mybatis-plus，Shiro，JWT。强大的代码生成器让前后端代码一键生成，无需写任何代码! 引领新低代码开发模式OnlineCoding->代码生成->手工MERGE，帮助Java项目解决70%重复工作，让开发更关注业务，既能快速提高开发效率，帮助公司节省成本，同时又不失灵活性。

JEECG BOOT 低代码开发平台（前后端分离版本） 当前最新版本： 2.4.2（发布日期：2021-01-26） 项目介绍： Java Low Code Platform for Enterprise web applications JeecgBoot 是一款基于代码生成器的低代码开发平台！前

Dec 31, 2022
Comments
  • Potential security vulnerability in the zstd C library.Can you help upgrade to patch versions?

    Potential security vulnerability in the zstd C library.Can you help upgrade to patch versions?

    Hi, @eyalschneider, I'd like to report a vulnerability issue in org.pipecraft.pipes:pipes-core:0.9.1.

    Issue Description

    I noticed that org.pipecraft.pipes:pipes-core:0.9.1 directly depends on com.github.luben:zstd-jni:v1.4.4-2 in the pom. However, as shown in the following dependency graph, com.github.luben:zstd-jni:v1.4.4-2 sufferes from the vulnerability which the C library zstd(version:1.4.4) exposed: CVE-2021-24032.

    Dependency Graph between Java and Shared Libraries

    image (12)

    Suggested Vulnerability Patch Versions

    com.github.luben:zstd-jni:v1.4.9-1 (>=v1.4.9-1) has upgraded this vulnerable C library zstd to the patch version 1.4.9.

    Java build tools cannot report vulnerable C libraries, which may induce potential security issues to many downstream Java projects. Could you please upgrade this vulnerable dependency?

    Thanks for your help~ Best regards, Helen Parr

    opened by HelenParr 1
  • Register in Maven Central

    Register in Maven Central

    We would like to add the following maven projects to maven central repo:

    • pipes-core
    • pipes-google-cs
    • pipes-google-bq
    • pipes-amazon-s3

    Their versions should be aligned

    opened by eyalschneider 0
Releases(v0.92)

  • v0.92(May 17, 2022)

    Release notes

    • Fixed a possible vulnerability by upgrading zstd dependency version (see https://nvd.nist.gov/vuln/detail/CVE-2021-24032)
    • Added Protocol Buffers encoder/decoder
    • LocalDiskBucket - fixed bug which made the class unusable for some operating systems
    • DistributedShufflerPipe - now dynamically decides which event loop mechanism to use based on the OS
    • Fixed concurrency bug in AsyncCollectionWriterPipe
    • Implemented MapWriterPipe and AsyncMapWriterPipe
    • Improved code documentation and made it more Javadoc compatible

    This release matches the following artifacts available in Maven Central:

    <dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-core</artifactId>
  <version>0.92</version>
</dependency>

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-google-cs</artifactId>
  <version>0.92</version>
</dependency>

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-google-bq</artifactId>
  <version>0.92</version>
</dependency>

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-amazon-s3</artifactId>
  <version>0.92</version>
</dependency>
    Source code(tar.gz)
    Source code(zip)
    pipes-amazon-s3-0.92-sources.jar(6.84 KB)
    pipes-core-0.92-sources.jar(271.79 KB)
    pipes-google-bq-0.92-sources.jar(36.95 KB)
    pipes-google-cs-0.92-sources.jar(18.87 KB)

  • v0.91(Jan 23, 2022)

    Release notes

    • Coding class - more serialization/deserialization utility functions
    • FileUtils - Additional file deletion methods
    • TxtCodecFactory/TxtEncoderFactory/TxtDecoderFactory - Added a simple identity encoder/decoder
    • Pipes relying on queues don't require the end/error markers anymore
    • QueuePipe - New synchronous intermediate pipe with queue support
    • SortPipe - Fixed a minor memory leak
    • AsyncSharderByHashPipe - Additional constructors for improved sharding flexibility

    This release matches the following artifacts available in Maven Central:

    <dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-core</artifactId>
  <version>0.91</version>
</dependency>

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-google-cs</artifactId>
  <version>0.91</version>
</dependency>

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-google-bq</artifactId>
  <version>0.91</version>
</dependency>

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-amazon-s3</artifactId>
  <version>0.91</version>
</dependency>
    Source code(tar.gz)
    Source code(zip)
    pipes-amazon-s3-0.91-sources.jar(6.78 KB)
    pipes-core-0.91-sources.jar(263.32 KB)
    pipes-google-bq-0.91-sources.jar(36.62 KB)
    pipes-google-cs-0.91-sources.jar(18.73 KB)

  • v0.9(Aug 2, 2021)

    This release matches the following artifacts available in Maven Central:

    <dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-core</artifactId>
  <version>0.9</version>
</dependency>

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-google-cs</artifactId>
  <version>0.9</version>
</dependency>

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-google-bq</artifactId>
  <version>0.9</version>
</dependency>

<dependency>
  <groupId>org.pipecraft.pipes</groupId>
  <artifactId>pipes-amazon-s3</artifactId>
  <version>0.9</version>
</dependency>
    Source code(tar.gz)
    Source code(zip)
    pipes-amazon-s3-0.9-sources.jar(6.76 KB)
    pipes-core-0.9-sources.jar(260.24 KB)
    pipes-google-bq-0.9-sources.jar(36.61 KB)
    pipes-google-cs-0.9-sources.jar(18.72 KB)
Owner
null
GitHub
An extensible Java framework for building XML and non-XML streaming applications

Smooks Framework This is the Git source code repository for the Smooks Project. Build Status Building Pre-requisites JDK 8 Apache Maven 3.2.x Maven gi

Smooks Framework 353 Dec 1, 2022
fastutil extends the Java Collections Framework by providing type-specific maps, sets, lists and queues.

fastutil is a collection of type-specific Java classes that extend the Java Collections Framework by providing several containers, such as maps, sets, lists and prority queues, implementing the interfaces of the java.util package; it also provides big (64-bit) arrays, sets, lists, and fast, practical I/O classes for binary and text files.

Sebastiano Vigna 1.5k Jan 7, 2023
Auto Code Audit Framework for Java

前言 笔者最近在研究java自动化代码审计这方面的内容，也看了一些相关的文章，其中主要是跟着4ra1n师傅的文章进行学习的。目前学到的有两种自动化审计思路，一是AST，二是ASM。前者基于java源代码，后者基于字节码。

r2 61 Nov 30, 2022
A spring cloud infrastructure provides various of commonly used cloud components and auto-configurations for high project consistency

A spring cloud infrastructure provides various of commonly used cloud components and auto-configurations for high project consistency.

Project-Hephaestus 2 Feb 8, 2022
Encog java core Apache 2 Encog java core Encog is an advanced machine learning framework that supports a variety of advanced algorithms, as well as support classes to normalize and process data. Machine learning algorithms such as Support Vector Machines, Artificial Neural Networks, Genetic Programming, Bayesian Networks, Hidden Markov Models, Genetic Programming and Genetic Algorithms are supported. License: Apache 2 , .

Encog Machine Learning Framework Encog is a pure-Java/C# machine learning framework that I created back in 2008 to support genetic programming, NEAT/H

Jeff Heaton 739 Dec 17, 2022
Core part of Jackson that defines Streaming API as well as basic shared abstractions

Overview This project contains core low-level incremental ("streaming") parser and generator abstractions used by Jackson Data Processor. It also incl

FasterXML, LLC 2.1k Jan 1, 2023
JPF is an extensible software analysis framework for Java bytecode. jpf-core is the basis for all JPF projects; you always need to install it. It contains the basic VM and model checking infrastructure, and can be used to check for concurrency defects like deadlocks, and unhandled exceptions like NullPointerExceptions and AssertionErrors.

Java Pathfinder (JPF) An extensible software model checking framework for Java bytecode programs General Information about JPF All the latest developm

Java Pathfinder 426 Dec 25, 2022
WaterCore - It's a Core, for SpigotPlugins, that contains some other API's

WaterCore - It's a Core, for SpigotPlugins, that contains some other API's (From other Developers)

RDProject 3 May 9, 2022
Core for open source libraries, included some important Classes for those libs.

OpenSource Core You could also read the CHINESE version of README This is a very useful Java class library. In this update, we have merged the origina

Theodore Hills 10 Nov 16, 2022
Zero is a core test automation project that can be used as a basis for any kind of test automation project (API, Browser, Mobile)

Zero Zero is our feature rich, core test automation framework, that can be used as an underlying automation framework for any/and all kind of test aut

Pramod Kumar Yadav 10 Dec 16, 2022