Dremio - the missing link in modern data

Overview

Dremio

Dremio enables organizations to unlock the value of their data.

Documentation

Documentation is available at https://docs.dremio.com.

Quickstart: How to build and run Dremio

(a) Prerequisites

  • JDK 8 (OpenJDK or Oracle)
  • (Optional) Maven 3.3.9 or later (using Homebrew: brew install maven)

Run the following commands to verify that you have the correct versions of Maven and JDK installed:

java -version
mvn --version

(b) Clone the Repository

git clone https://github.com/dremio/dremio-oss.git dremio

(c) Build the Code

cd dremio
mvn clean install -DskipTests (or ./mvnw clean install -DskipTests if maven is not installed on the machine)

The "-DskipTests" option skips most of the tests. Running all tests takes a long time.

(d) Run/Install

Run

distribution/server/target/dremio-community-{DREMIO_VERSION}/dremio-community-{DREMIO_VERSION}/bin/dremio start

OR to start a server with a default user (dremio/dremio123)

mvn compile exec:exec -pl dac/daemon

Once run, the UI is accessible at:

http://localhost:9047

Production Install

(1) Unpack the tarball to install.
mkdir /opt/dremio
tar xvzf distribution/server/target/*.tar.gz --strip=1 -C /opt/dremio
(2) Start Dremio Embedded Mode
cd /opt/dremio
bin/dremio

OSS Only

To have the best possible experience with Dremio, we include a number of dependencies when building Dremio that are distributed under non-oss free (as in beer) licenses. Examples include drivers for major databases such as Oracle Database, Microsoft SQL Server, MySQL as well as enhancements to improve source pushdowns and thread scheduling. If you'd like to only include dependencies with OSS licenses, Dremio will continue to work but some features will be unavailable (such as connecting to databases that rely on these drivers).

To build dremio with only OSS dependencies, you can add the following option to your Maven commandline: -Ddremio.oss-only=true

The distribution directory will be distribution/server/target/dremio-oss-{DREMIO_VERSION}/dremio-oss-{DREMIO_VERSION}

Questions?

If you have questions, please post them on https://community.dremio.com.

Comments
  • Bump async from 2.6.3 to 2.6.4 in /dac/ui-lib

    Bump async from 2.6.3 to 2.6.4 in /dac/ui-lib

    Bumps async from 2.6.3 to 2.6.4.

    Changelog

    Sourced from async's changelog.

    v2.6.4

    • Fix potential prototype pollution exploit (#1828)
    Commits
    Maintainer changes

    This version was pushed to npm by hargasinski, a new releaser for async since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 1
  • Bump eventsource from 1.0.7 to 1.1.1 in /dac/ui-lib

    Bump eventsource from 1.0.7 to 1.1.1 in /dac/ui-lib

    Bumps eventsource from 1.0.7 to 1.1.1.

    Changelog

    Sourced from eventsource's changelog.

    1.1.1

    • Do not include authorization and cookie headers on redirect to different origin (#273 Espen Hovlandsdal)

    1.1.0

    • Improve performance for large messages across many chunks (#130 Trent Willis)
    • Add createConnection option for http or https requests (#120 Vasily Lavrov)
    • Support HTTP 302 redirects (#116 Ryan Bonte)
    • Prevent sequential errors from attempting multiple reconnections (#125 David Patty)
    • Add new to correct test (#111 Stéphane Alnet)
    • Fix reconnections attempts now happen more than once (#136 Icy Fish)
    Commits
    • aa7a408 1.1.1
    • 56d489e chore: rebuild polyfill
    • 4a951e5 docs: update history for 1.1.1
    • f9f6416 fix: strip sensitive headers on redirect to different origin
    • 9dd0687 1.1.0
    • 49497ba Update history for 1.1.0 (#146)
    • 3a38537 Update history for #136
    • 46fe04e Merge pull request #136 from icy-fish/master
    • 9a4190f Fix issue: reconnection only happends for 1 time after connection drops
    • 61e1b19 test: destroy both proxied request and response on close
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 1
  • Bump gson from 2.8.5 to 2.8.9 in /plugins/elasticsearch

    Bump gson from 2.8.5 to 2.8.9 in /plugins/elasticsearch

    Bumps gson from 2.8.5 to 2.8.9.

    Release notes

    Sourced from gson's releases.

    Gson 2.8.9

    • Make OSGi bundle's dependency on sun.misc optional (#1993).
    • Deprecate Gson.excluder() exposing internal Excluder class (#1986).
    • Prevent Java deserialization of internal classes (#1991).
    • Improve number strategy implementation (#1987).
    • Fix LongSerializationPolicy null handling being inconsistent with Gson (#1990).
    • Support arbitrary Number implementation for Object and Number deserialization (#1290).
    • Bump proguard-maven-plugin from 2.4.0 to 2.5.1 (#1980).
    • Don't exclude static local classes (#1969).
    • Fix RuntimeTypeAdapterFactory depending on internal Streams class (#1959).
    • Improve Maven build (#1964).
    • Make dependency on java.sql optional (#1707).

    Gson 2.8.8

    • Fixed issue with recursive types (#1390).
    • Better behaviour with Java 9+ and Unsafe if there is a security manager (#1712).
    • EnumTypeAdapter now works better when ProGuard has obfuscated enum fields (#1495).
    Changelog

    Sourced from gson's changelog.

    Version 2.8.9

    • Make OSGi bundle's dependency on sun.misc optional (#1993).
    • Deprecate Gson.excluder() exposing internal Excluder class (#1986).
    • Prevent Java deserialization of internal classes (#1991).
    • Improve number strategy implementation (#1987).
    • Fix LongSerializationPolicy null handling being inconsistent with Gson (#1990).
    • Support arbitrary Number implementation for Object and Number deserialization (#1290).
    • Bump proguard-maven-plugin from 2.4.0 to 2.5.1 (#1980).
    • Don't exclude static local classes (#1969).
    • Fix RuntimeTypeAdapterFactory depending on internal Streams class (#1959).
    • Improve Maven build (#1964).
    • Make dependency on java.sql optional (#1707).

    Version 2.8.8

    • Fixed issue with recursive types (#1390).
    • Better behaviour with Java 9+ and Unsafe if there is a security manager (#1712).
    • EnumTypeAdapter now works better when ProGuard has obfuscated enum fields (#1495).

    Version 2.8.7

    • Fixed ISO8601UtilsTest failing on systems with UTC+X.
    • Improved javadoc for JsonStreamParser.
    • Updated proguard.cfg (#1693).
    • Fixed IllegalStateException in JsonTreeWriter (#1592).
    • Added JsonArray.isEmpty() (#1640).
    • Added new test cases (#1638).
    • Fixed OSGi metadata generation to work on JavaSE < 9 (#1603).

    Version 2.8.6

    2019-10-04 GitHub Diff

    • Added static methods JsonParser.parseString and JsonParser.parseReader and deprecated instance method JsonParser.parse
    • Java 9 module-info support
    Commits
    • 6a368d8 [maven-release-plugin] prepare release gson-parent-2.8.9
    • ba96d53 Fix missing bounds checks for JsonTreeReader.getPath() (#2001)
    • ca1df7f #1981: Optional OSGi bundle's dependency on sun.misc package (#1993)
    • c54caf3 Deprecate Gson.excluder() exposing internal Excluder class (#1986)
    • e6fae59 Prevent Java deserialization of internal classes (#1991)
    • bda2e3d Improve number strategy implementation (#1987)
    • cd748df Fix LongSerializationPolicy null handling being inconsistent with Gson (#1990)
    • fe30b85 Support arbitrary Number implementation for Object and Number deserialization...
    • 1cc1627 Fix incorrect feature request template label (#1982)
    • 7b9a283 Bump bnd-maven-plugin from 5.3.0 to 6.0.0 (#1985)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies java 
    opened by dependabot[bot] 1
  • Bump marked from 0.7.0 to 4.0.10 in /dac/ui

    Bump marked from 0.7.0 to 4.0.10 in /dac/ui

    Bumps marked from 0.7.0 to 4.0.10.

    Release notes

    Sourced from marked's releases.

    v4.0.10

    4.0.10 (2022-01-13)

    Bug Fixes

    • security: fix redos vulnerabilities (8f80657)

    v4.0.9

    4.0.9 (2022-01-06)

    Bug Fixes

    v4.0.8

    4.0.8 (2021-12-19)

    Bug Fixes

    v4.0.7

    4.0.7 (2021-12-09)

    Bug Fixes

    v4.0.6

    4.0.6 (2021-12-02)

    Bug Fixes

    v4.0.5

    4.0.5 (2021-11-25)

    Bug Fixes

    • table after paragraph without blank line (#2298) (5714212)

    v4.0.4

    4.0.4 (2021-11-19)

    ... (truncated)

    Commits
    • ae01170 chore(release): 4.0.10 [skip ci]
    • fceda57 🗜️ build [skip ci]
    • 8f80657 fix(security): fix redos vulnerabilities
    • c4a3ccd Merge pull request from GHSA-rrrm-qjm4-v8hf
    • d7212a6 chore(deps-dev): Bump jasmine from 4.0.0 to 4.0.1 (#2352)
    • 5a84db5 chore(deps-dev): Bump rollup from 2.62.0 to 2.63.0 (#2350)
    • 2bc67a5 chore(deps-dev): Bump markdown-it from 12.3.0 to 12.3.2 (#2351)
    • 98996b8 chore(deps-dev): Bump @​babel/preset-env from 7.16.5 to 7.16.7 (#2353)
    • ebc2c95 chore(deps-dev): Bump highlight.js from 11.3.1 to 11.4.0 (#2354)
    • e5171a9 chore(release): 4.0.9 [skip ci]
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 1
  • Bump lodash from 4.17.19 to 4.17.21 in /dac/ui

    Bump lodash from 4.17.19 to 4.17.21 in /dac/ui

    Bumps lodash from 4.17.19 to 4.17.21.

    Commits
    • f299b52 Bump to v4.17.21
    • c4847eb Improve performance of toNumber, trim and trimEnd on large input strings
    • 3469357 Prevent command injection through _.template's variable option
    • ded9bc6 Bump to v4.17.20.
    • 63150ef Documentation fixes.
    • 00f0f62 test.js: Remove trailing comma.
    • 846e434 Temporarily use a custom fork of lodash-cli.
    • 5d046f3 Re-enable Travis tests on 4.17 branch.
    • aa816b3 Remove /npm-package.
    • See full diff in compare view
    Maintainer changes

    This version was pushed to npm by bnjmnt4n, a new releaser for lodash since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 1
  • Bump commons-io from 2.4 to 2.7 in /tools/fmpp-maven-plugin

    Bump commons-io from 2.4 to 2.7 in /tools/fmpp-maven-plugin

    Bumps commons-io from 2.4 to 2.7.

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump libthrift from 0.9.3-2dremio to 0.12.0 in /contrib/hive2-exec-shade

    Bump libthrift from 0.9.3-2dremio to 0.12.0 in /contrib/hive2-exec-shade

    Bumps libthrift from 0.9.3-2dremio to 0.12.0.

    Release notes

    Sourced from libthrift's releases.

    Version 0.12.0

    Thrift v0.12.0

    New Languages

    * Common LISP (cl)
    * Swift
    * Typescript (nodets)
    

    Complete Release Notes

    https://github.com/apache/thrift/blob/0.12.0/CHANGES

    Version 0.9.3.1

    This release is a backport of the security fix for CVE-2018-1320 as documented in THRIFT-4506. The only code change is in Java, and a 0.9.3-1 package was released to Maven Central.

    This is marked in GitHub as a pre-release so that it does not become the "latest" release.

    Changelog

    Sourced from libthrift's changelog.

    0.12.0

    Released 2019-JAN-04

    New Languages

    • Common LISP (cl)
    • Swift
    • Typescript (nodets)

    Deprecated Languages

    • C++03/C++98 (move to C++11)
    • Cocoa (move to Swift)

    Breaking Changes (since 0.11.0)

    • THRIFT-4529 - Rust enum variants are now camel-cased instead of uppercased to conform to Rust naming conventions
    • THRIFT-4448 - Support for golang 1.6 and earlier has been dropped.
    • THRIFT-4474 - PHP now uses the PSR-4 loader by default instead of class maps.
    • THRIFT-4532 - method signatures changed in the compiler's t_oop_generator.
    • THRIFT-4648 - The C (GLib) compiler's handling of namespaces has been improved.

    Known Issues (Blocker or Critical)

    • THRIFT-4037 - build: use a single build system for thrift
    • THRIFT-4119 - build: bootstrap.sh is missing from source tarball
    • THRIFT-3289 - csharp: socket exhaustion in csharp implementation
    • THRIFT-3029 - cocoa: Getters for fields defined with uppercase names do not work
    • THRIFT-3325 - cocoa: Extended services aren't subclasses in generated Cocoa
    • THRIFT-4116 - cocoa: Thrift de-capitalizes the name of IsSet property in Cocoa
    • THRIFT-3877 - cpp: the http implementation is not standard; interop with other languages is spotty at best
    • THRIFT-4180 - cpp: Impossible to build Thrift C++ library for Android (NDK)
    • THRIFT-4384 - cpp: Using multiple async services simultaneously is not thread-safe
    • THRIFT-3108 - haskell: Defaulted struct parameters on a service generates invalid Haskell
    • THRIFT-3990 - nodejs: Exception swallowed by deserialization function
    • THRIFT-4214 - nodejs: map key treated as hex value in JavaScript
    • THRIFT-4602 - nodejs: ERROR in ./node_modules/thrift/lib/nodejs/lib/thrift/connection.js Module not found: Error: Can't resolve 'child_process'
    • THRIFT-4639 - nodejs: Sequence numbering for multiplexed protocol broken
    • THRIFT-1310 - php: sequence and reconnection management issues
    • THRIFT-1538 - php: Error during deserialization int64 on 32-bit architecture
    • THRIFT-1580 - php: thrift type i64 java to php serialize/deserealize not working
    • THRIFT-1950 - php: PHP gets stuck in infinite loop
    • THRIFT-2954 - python: sending int or float in a double field breaks the connection
    • THRIFT-4080 - python: unix sockets can get stuck forever
    • THRIFT-4281 - python: generated code is out of order and causes load issues
    • THRIFT-4677 - py3: UnicodeDecideError in Python3

    Build Process

    • THRIFT-4067 - Windows thrift compiler distributed on the apache web site has runtime dependencies
    • THRIFT-4308 - D language docker images need demios for libevent and openssl fixed to re-enable make cross on dlang
    • THRIFT-4579 - Use Ubuntu Bionic (18.04 LTS) for CI builds instead of Artful (17.10)
    • THRIFT-4508 - Define CI operating system coverage rules for the project and (hopefully) simplify CI a little more
    • THRIFT-4397 - ubuntu install instructions broken on 16.04
    ... (truncated)
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Upgrade MongoDB Java Driver version

    Upgrade MongoDB Java Driver version

    Problem: Unable to authenticate to mongodb version 4.0.9 hosted locally with docker, which uses SCRAM-SHA-256 as the only authentication method and no SSL certificate. Note: The problem does not arise if you decide to use SCRAM-SHA-1

    I propose to switch to a version of the Mongo Java driver greater than or equal to 3.8.0. Dremio 4.1.4 uses a driver version of 3.6.4. From my tests I was able to draw up a list of drivers that are compatible or not with SCRAM-SHA-256 authentication:

    non-working driver versions: 3.5.0, 3.6.4, 3.7.0, 3.7.0-rc0, 3.7.1

    working driver versions: 3.8.0, 3.8.1, 3.8.2, 3.9.0, 3.10.2, 3.11.2, 3.12.1

    Link to the java driver repository

    These are the steps I made to locally test the authentication towards MongoDB:

    • I used docker-compose up (using the docker-compose.yml contained in the attachment docker-compose.zip)
    • used Docui to easily access the logs and to run bash on mongodb and dremio:
    • I run bash on the mongo container, I type mongo admin
    • then I create two users (note: I can't create a user with SCRAM-SHA-1 authentication mechanism because I imposed it in the docker-compose.yml file)

    db.createUser({user: 'ian256',pwd: 'ian256',mechanisms: [ "SCRAM-SHA-256" ],passwordDigestor : "server",roles : ["root"] })

    db.createUser({user: 'ian',pwd: 'ian',passwordDigestor : "server",roles : ["root"] })

    To see that dremio 4.1.4 (which uses the 3.6.4 driver) does not authenticate, just go to localhost: 9047 and try adding MongoDB as the source, inserting the IP used by the docker as the Host, as the user and password one of the two (ian or ian256) and note that both the dremio and mongo logs give errors(in my case, from mongo logs:

    2020-02-18T12:05:51.502+0000 I NETWORK [conn19] received client metadata from 172.29.0.2:48696 conn19: { driver: { name: "mongo-java-driver", version: "3.6.4" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "5.4.14-2-MANJARO" }, platform: "Java/Oracle Corporation/1.8.0_242-b08" } 2020-02-18T12:05:51.508+0000 I ACCESS [conn19] SASL SCRAM-SHA-1 authentication failed for ian1 on admin from client 172.29.0.2:48696 ; BadValue: SCRAM-SHA-1 authentication is disabled 2020-02-18T12:05:51.511+0000 I NETWORK [conn19] end connection 172.29.0.2:48696 (1 connection now open)

    To try Dremio with a driver with version 3.8.0 or later, I downloaded dremio from here, I unpacked it, I went to the dremio-community-4.1.4-202001240912140359-a90eb503/jars/3rdparty directory, I eliminated mongo-java-driver-3.6.4.jar and I put mongo-java-driver-3.8.0.jar, after which I started dremio with dremio-community-4.1.4-202001240912140359-a90eb503/bin/dremio start

    To see that instead dremio 4.1.4 (which now uses the 3.8.0 driver) now manages to authenticate, just go to localhost: 9047 and try adding MongoDB as the source, inserting the IP used by the docker as the Host, as the user and password one of the two (ian or ian256) and note that both the dremio and mongo logs do not give errors(in my case, from mongo logs:

    2020-02-18T12:08:07.964+0000 I NETWORK [conn22] received client metadata from 172.29.0.1:36498 conn22: { driver: { name: "mongo-java-driver", version: "3.8.0-beta3-44-g1ff4ce53c-dirty" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "5.4.14-2-MANJARO" }, platform: "Java/Oracle Corporation/1.8.0_231-b11" } 2020-02-18T12:08:08.086+0000 I NETWORK [listener] connection accepted from 172.29.0.1:36500 #23 (3 connections now open)

    I hope I have given enough details on the issue.

    opened by AndreaTosti 1
  • Create test that prints only Gandiva functions from Expression Registry

    Create test that prints only Gandiva functions from Expression Registry

    Requester

    This task was requested by Vivekanand Vellanki from Dremio team.

    Contribution

    This Pull Request is a Simbiose Team contribution, adding a simple Java unit test which prints only the Gandiva available functions from Expression Registry.

    Checklist

    • [x] Implement test method for printing only Gandiva functions;
    • [x] Sign the CLA for contributing on Dremio OSS
    • [x] Send the pull request;
    opened by jpedroantunes 0
  • Upgrade to Jackson BOM 2.13.4.20221013 to fix CVE-2022-42003 and CVE-2022-42004

    Upgrade to Jackson BOM 2.13.4.20221013 to fix CVE-2022-42003 and CVE-2022-42004

    We should update to jackson-databind 2.13.4.2 to fix couple of CVEs:

    • https://nvd.nist.gov/vuln/detail/CVE-2022-42003
    • https://nvd.nist.gov/vuln/detail/CVE-2022-42004
    opened by jbonofre 0
  • "Force" locale in joda time common test to work with any user setting

    DayOfWeekFromSundayDateTimeFieldTest can fail for some user, depending the locale settings. Basically, the tests assume the Locale is set to US.

    As a result, the maven build can fail:

    [ERROR] Failures: 
    [ERROR] org.joda.time.chrono.DayOfWeekFromSundayDateTimeFieldTest.getAsShortText
    [ERROR]   Run 1: DayOfWeekFromSundayDateTimeFieldTest.getAsShortText:49
    [ERROR]   Run 2: DayOfWeekFromSundayDateTimeFieldTest.getAsShortText:49
    [ERROR]   Run 3: DayOfWeekFromSundayDateTimeFieldTest.getAsShortText:49
    [ERROR]   Run 4: DayOfWeekFromSundayDateTimeFieldTest.getAsShortText:49
    [INFO] 
    [ERROR] org.joda.time.chrono.DayOfWeekFromSundayDateTimeFieldTest.getAsShortTextFieldValue
    [ERROR]   Run 1: DayOfWeekFromSundayDateTimeFieldTest.getAsShortTextFieldValue:63
    [ERROR]   Run 2: DayOfWeekFromSundayDateTimeFieldTest.getAsShortTextFieldValue:63
    [ERROR]   Run 3: DayOfWeekFromSundayDateTimeFieldTest.getAsShortTextFieldValue:63
    [ERROR]   Run 4: DayOfWeekFromSundayDateTimeFieldTest.getAsShortTextFieldValue:63
    [INFO] 
    [ERROR] org.joda.time.chrono.DayOfWeekFromSundayDateTimeFieldTest.getAsText
    [ERROR]   Run 1: DayOfWeekFromSundayDateTimeFieldTest.getAsText:42
    [ERROR]   Run 2: DayOfWeekFromSundayDateTimeFieldTest.getAsText:42
    [ERROR]   Run 3: DayOfWeekFromSundayDateTimeFieldTest.getAsText:42
    [ERROR]   Run 4: DayOfWeekFromSundayDateTimeFieldTest.getAsText:42
    [INFO] 
    [ERROR] org.joda.time.chrono.DayOfWeekFromSundayDateTimeFieldTest.getAsTextFieldValue
    [ERROR]   Run 1: DayOfWeekFromSundayDateTimeFieldTest.getAsTextFieldValue:56
    [ERROR]   Run 2: DayOfWeekFromSundayDateTimeFieldTest.getAsTextFieldValue:56
    [ERROR]   Run 3: DayOfWeekFromSundayDateTimeFieldTest.getAsTextFieldValue:56
    [ERROR]   Run 4: DayOfWeekFromSundayDateTimeFieldTest.getAsTextFieldValue:56
    

    This PR sets the Locale to US, providing successful test whatever is the user Locale.

    opened by jbonofre 0
  • bugfix 'win error on git clone' by rename file

    bugfix 'win error on git clone' by rename file

    win error on git clone because win does not give store file with file name which contains symbol '?'

    error:

    git clone https://github.com/dremio/dremio-oss.git dremio error: invalid path 'dac/backend/src/test/resources/testfiles/file_with_?.json' fatal: unable to checkout working tree warning: Clone succeeded, but checkout failed.

    opened by satonin-v-yu 0
  • Bump terser from 4.8.0 to 4.8.1 in /dac/ui-lib/example

    Bump terser from 4.8.0 to 4.8.1 in /dac/ui-lib/example

    Bumps terser from 4.8.0 to 4.8.1.

    Changelog

    Sourced from terser's changelog.

    v4.8.1 (backport)

    • Security fix for RegExps that should not be evaluated (regexp DDOS)
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 0
  • Bump commons-io from 2.6 to 2.7 in /services/datastore

    Bump commons-io from 2.6 to 2.7 in /services/datastore

    Bumps commons-io from 2.6 to 2.7.

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies java 
    opened by dependabot[bot] 0
A modern I/O library for Android, Kotlin, and Java.

Okio See the project website for documentation and APIs. Okio is a library that complements java.io and java.nio to make it much easier to access, sto

Square 8.2k Nov 30, 2022
High Performance data structures and utility methods for Java

Agrona Agrona provides a library of data structures and utility methods that are a common need when building high-performance applications in Java. Ma

Real Logic 2.4k Dec 5, 2022
Clojure's data structures modified for use outside of Clojure

This library has been extracted from the master branch of Clojure (http://clojure.org) version 1.5.1 (as of October 2013) http://github.com/richhick

Karl Krukow 221 Oct 6, 2022
Eclipse Collections is a collections framework for Java with optimized data structures and a rich, functional and fluent API.

English | 中文 | Deutsch | Español | Ελληνικά | Français | 日本語 | Norsk (bokmål) | Português-Brasil | Русский | हिंदी Eclipse Collections is a comprehens

Eclipse Foundation 2.1k Dec 8, 2022
Reading Dalta Lake data from Beam

Reading Delta Lake Data from Beam General Info: All files, except org.apache.beam.sdk.io.DeltaFileIO are from Daltalake Standalone Reader. I was not a

Michael 6 Nov 21, 2022
Table-Computing (Simplified as TC) is a distributed light weighted, high performance and low latency stream processing and data analysis framework. Milliseconds latency and 10+ times faster than Flink for complicated use cases.

Table-Computing Welcome to the Table-Computing GitHub. Table-Computing (Simplified as TC) is a distributed light weighted, high performance and low la

Alibaba 34 Oct 14, 2022
An embedded database implemented in pure java based on bitcask which is a log-structured hash table for K/V Data.

Baka Db An embedded database implemented in pure java based on bitcask which is a log-structured hash table for K/V Data. Usage import cn.ryoii.baka.B

ryoii 3 Dec 20, 2021
Jalgorithm is an open-source Java library which has implemented various algorithms and data structure

We loved Java and algorithms, so We made Jalgorithm ❤ Jalgorithm is an open-source Java library which has implemented various algorithms and data stru

Muhammad Karbalaee 35 Nov 18, 2022
BioJava is an open-source project dedicated to providing a Java framework for processing biological data.

Welcome to BioJava is an open-source project dedicated to providing a Java framework for processing biological data. It provides analytical and statis

BioJava 512 Nov 21, 2022
The Java collections framework provides a set of interfaces and classes to implement various data structures and algorithms.

Homework #14 Table of Contents General Info Technologies Used Project Status Contact General Information Homework contains topics: Sorting an ArrayLis

Mykhailo 1 Feb 12, 2022
SWE5003 - Achitecting Real Time Systems for Data Processing - Code Base

ARTS2022 SWE5003 - Achitecting Real Time Systems for Data Processing (ISS NUS Offering) - Code Base This module is part of the ISS MTech Graduate Cert

Suria R Asai 5 Apr 2, 2022
Download or repost public instagram posts easily by selecting "Copy Link" in Instagram.

Insta Save and Repost Description Download or repost public instagram posts easily by selecting "Copy Link" in Instagram. Screenshots Features Downloa

null 1 Jan 21, 2022
JCTools - Concurrency tools currently missing from the JDK.

JCTools Java Concurrency Tools for the JVM. This project aims to offer some concurrent data structures currently missing from the JDK: SPSC/MPSC/SPMC/

null 3.1k Dec 6, 2022
The missing bridge between Java and native C++

JavaCPP Commercial support: Introduction JavaCPP provides efficient access to native C++ inside Java, not unlike the way some C/C++ compilers interact

Bytedeco 4k Dec 7, 2022
Resconstruct is a java library to infer missing information vectors of java classes.

Reconstruct Resconstruct is a java library to infer missing information vectors of java classes. Features Phantom classes Inheritance solving Dummy fi

Nowilltolife 14 Nov 17, 2022
A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.

Apache Gobblin Apache Gobblin is a highly scalable data management solution for structured and byte-oriented data in heterogeneous data ecosystems. Ca

The Apache Software Foundation 2.1k Nov 29, 2022
A scientific charting library focused on performance optimised real-time data visualisation at 25 Hz update rates for data sets with a few 10 thousand up to 5 million data points.

ChartFx ChartFx is a scientific charting library developed at GSI for FAIR with focus on performance optimised real-time data visualisation at 25 Hz u

GSI CS-CO/ACO 375 Nov 27, 2022