Time series monitoring and alerting platform.

Overview

Argus Build Status Static Analysis argus-sdk - Maven Central

Argus is a time-series monitoring and alerting platform. It consists of discrete services to configure alerts, ingest and transform metrics & events, send notifications, create namespaces, and to both establish and enforce policies and quotas for usage.

Its architecture allows any and all of these services to be retargeted to new technology as it becomes available, with little to no impact on the users.

To find out more see the wiki and check out the release notes.

Argus UI

Building Argus

Installing The Resource Filters

Argus uses the argus-build.properties file as a resource filter for the build and all the module builds. After you clone the project for the first time, or after you change this file, you must create and install the dependency jars which will contain these filters. Those dependency jars are then pulled in by the modules, expanded and have their values applied to the module specific builds. Luckily, it's a straightforward operation. Just execute the following command from within the parent project, after you first clone the project or after you update the argus-build.properties file.

mvn -DskipTests=true -DskipDockerBuild --non-recursive install

Running The Unit Tests

Once the resource filters are installed, you can run unit tests. Running the unit tests doesn't require any changes to the argus-build.properties file. Just install the resource filters and execute the test goal.

mvn test

Only the unit tests are run by codecov.io and as such, the coverage reported by it is significantly less than the coverage obtained by running the full test suite.

Running The Integration Tests (Deprecated)

Currently the integration tests are non-functional but are being retained in case we want to resurrect them.

The integration tests for Argus use the LDAPAuthService implementation of the AuthService interface and the DefaultTSDBService implementation of the TSDBService interface (which targets OpenTSDB). Additionally it uses the RedisCacheService implementation of the CacheService interface to facilitate integration testing of the BatchService. In order to run the integration tests you must update the argus-build.properties file to correctly setup the external LDAP you'll be testing against and the OpenTSDB endpoints to use as well as the Redis cluster. The snippet below shows the specific properties that should be modified in the argus-build.properties file. Of course, after you make these updates, you must re-install the resource filter dependencies as described above and execute the clean goal, before running the integration tests.

# The LDAP endpoint to use
service.property.auth.ldap.endpoint=ldaps://ldaps.yourdomain.com:636
# A list of comma separated search paths used to query the DN of users attempting to authenticate.
# This example lists two separate search bases.  One for users and one for service accounts.
service.property.auth.ldap.searchbase=OU=active,OU=user,DC=yourdomain,DC=com:OU=active,OU=robot,DC=yourdomain,DC=com
# This specifies of the DN for the privileged user that is used to bind and subsequently execute the search for user DN's
service.property.auth.ldap.searchdn=CN=argus_admin,OU=active,OU=user,DC=yourdomain,DC=com
# The password for the privileged user above.
service.property.auth.ldap.searchpwd=Argu5R0cks!
# The LDAP field with which the username provided during a login attempt, will be matched.
# This is used so Argus can obtain the DN for the user attempting to login, and subsequently attempt to bind as that user.
service.property.auth.ldap.usernamefield=sAMAccountName
# The TSDB read endpoint
service.property.tsdb.endpoint.read=http://readtsdb.yourdomain.com:4466
# The TSDB write endpoint
service.property.tsdb.endpoint.write=http://writetsdb.yourdomain.com:4477
# The Redis cache cluster information
service.property.cache.redis.cluster=redis0.mycompany.com:6379,redis1.mycompany.com:6389

Once the modifications have been made and the resource filters re-installed, you're ready to run the complete suite of tests, including the integration tests.

mvn verify

Generating Coverage Reports

Coverage is calculated everytime tests are run for all modules with the exception of ArgusWeb. In order to generate a coverage report for a module, just cd into the module subdirectory and run the report generation target.

mvn jacoco:report

Coverage reports are generated in the target/site/jacoco directory.

Deploying & Running Argus

Please see the wiki for information on how to deploy, configure and run Argus.

Comments
  • callback notifier

    callback notifier

    Hey guys,

    I'd like to introduce a new notification type. It is created to send a http request to a rest end point outside of argus.

    The notifier is configured by the alerts notification as usual. The notification is handled by the CallbackService. The configuration is build in Json format and defined as:

    {
      "uri": "<REST address>",
      "method": "<one of POST|PUT|GET|DELETE>",
      "header": { "key1": "value1", ...},
      "body": "<REST message or message template>"
      "template": "<template language or null>"
    }
    

    The REST message might be build of a template language, where the CallbackService subsitutes the template variables by the notification context specific values. I've choosen string template as the template language for the sample case. Substituteable variables are all DefaultAlertService.NotificationContext members. REST client is the apache HTTP client. The service manages a pool of shared clients.

    Note: The json expression is build based on the concatination of all notifications subscriptions (due to the database column lenth limit of 255 chars). We might think about an additional field. But reusing the subscriptions doesn't required a DB schema change.

    feature 
    opened by krasv 12
  • Dashboard decomposition and Chart APIs

    Dashboard decomposition and Chart APIs

    This pull request involves following changes

    • Changes made by @bsura related to adding ability to decompose dashboards into charts, preferences

    • Changes made my myself related to adding version field for dashboard and description field for chart

    cla:missing 
    opened by cannakula-sfdc 12
  • [Salesforce Migration] Warning

    [Salesforce Migration] Warning

    Salesforce is in the process of migrating all repositories from github.com/SalesforceEng to github.com/salesforce. As a part of this migration, this repository is slated to be moved as well. To request an exception or for any concerns contact @SalesforceEng/osscore or email [email protected]. There is no set date for the migration yet. For details on how GitHub repository transfers work, see here. Posted at 2016-08-23 22:45:38 PST.

    opened by cloudybot 10
  • Features aug 2019

    Features aug 2019

    Add Histogram support New Transforms Several bug fixes Upgrade kafka to 2.x Upgrade Elasticsearch to 6.x Add retention feature for schema docs Migrate annotation to elasticsearch Add pagerduty notifier Increase code coverage Enhance datalag detection and disabling of alerts Add Image chart feature to alert notifications

    opened by nkunal 9
  • Adding New Iterator Transform: foreach

    Adding New Iterator Transform: foreach

    Adding New Iterator Transform: foreach Syntax example:
    OLD: SUM(SCALE(-10d:M:cpu{device=*}:avg, -10d:M:count{device=*}:avg)) NEW: SUM(foreach(-10d:M:cpu{device=*}:avg,-10d:M:count{device=*}:avg,$device,$SCALE))

    Scope: Can apply on any existing TRANSFORM with their existing constants requirement if any.

    Detail: foreach is implemented against a new type "TRANSFORMITERATOR", which extends current type "TRANSFORM". This architecture allows "foreach" being treated same way in metricReader and factory. However, they are different in that:

    CURRENT TYPE Transform.transform: taking Array<Metric>, giving Array<Metric> NEW TYPE TransformIterator.iterate: taking Array<Metric>, giving Array<TransformExecutable>

    in above, each TransformExecutable is Array<Metric>, so they that can be consumed directly by any Transform.transform

    opened by aertoria 9
  • Add 'cts' field for scopemetricnames index | Add 'cts' field for scopenames index

    Add 'cts' field for scopemetricnames index | Add 'cts' field for scopenames index

    Add 'cts' field for scopenames index Add 'cts' field for scopemetricnames index Fix bug with alertdefinition logging. Enabling using scopeandmetric.index.

    opened by naveenreddykarri 8
  • D3 show full date range user defined in chart & fix chart not showing chart title

    D3 show full date range user defined in chart & fix chart not showing chart title

    • show full date range user defined in chart
    • fix chart not showing chart title
    d3 
    opened by prestonfff 8
  • Discovery Service fetch limit estimation

    Discovery Service fetch limit estimation

    Discovery Service fetch limit estimation: currently is based on the assumption of minute resolution and multiplies with time range. This causing lots of query been overestimated and therefore failed to fetch.

    To fix this, purposing, please first take a sample to determine the average resolution and then use it to decide the data volume that will be fetched.

    Let me know if you need more details.

    @saaja-sfdc @kgowdru

    question 
    opened by aertoria 8
  • Anomaly Detection Transforms

    Anomaly Detection Transforms

    Contains implementations and unit tests for two anomaly detection transforms based on the Gaussian (Normal) distribution:

    1. Probability density - ANOMALY_DENSITY(<time_series>)
    2. Z-score - ANOMALY_ZSCORE(<time_series>)
    enhancement 
    opened by shouvikmani 8
  • Call .travis.deploy.sh w/bash and use curly braces

    Call .travis.deploy.sh w/bash and use curly braces

    opened by justinharringa 7
  • Bump junit from 4.12 to 4.13.1

    Bump junit from 4.12 to 4.13.1

    Bumps junit from 4.12 to 4.13.1.

    Release notes

    Sourced from junit's releases.

    JUnit 4.13.1

    Please refer to the release notes for details.

    JUnit 4.13

    Please refer to the release notes for details.

    JUnit 4.13 RC 2

    Please refer to the release notes for details.

    JUnit 4.13 RC 1

    Please refer to the release notes for details.

    JUnit 4.13 Beta 3

    Please refer to the release notes for details.

    JUnit 4.13 Beta 2

    Please refer to the release notes for details.

    JUnit 4.13 Beta 1

    Please refer to the release notes for details.

    Commits
    • 1b683f4 [maven-release-plugin] prepare release r4.13.1
    • ce6ce3a Draft 4.13.1 release notes
    • c29dd82 Change version to 4.13.1-SNAPSHOT
    • 1d17486 Add a link to assertThrows in exception testing
    • 543905d Use separate line for annotation in Javadoc
    • 510e906 Add sub headlines to class Javadoc
    • 610155b Merge pull request from GHSA-269g-pwp5-87pp
    • b6cfd1e Explicitly wrap float parameter for consistency (#1671)
    • a5d205c Fix GitHub link in FAQ (#1672)
    • 3a5c6b4 Deprecated since jdk9 replacing constructor instance of Double and Float (#1660)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies java 
    opened by dependabot[bot] 0
  • [Snyk] Fix for 1 vulnerabilities

    [Snyk] Fix for 1 vulnerabilities

    Snyk has created this PR to fix one or more vulnerable packages in the `maven` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • pom.xml
      • ArgusWebServices/pom.xml

    Vulnerabilities that will be fixed

    With an upgrade:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 701/1000
    Why? Recently disclosed, Has a fix available, CVSS 8.1 | Deserialization of Untrusted Data
    SNYK-JAVA-COMFASTERXMLJACKSONCORE-1009829 | com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:
    2.9.5 -> 2.10.0
    com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:
    2.9.5 -> 2.10.0
    org.glassfish.jersey.media:jersey-media-json-jackson:
    2.26 -> 2.30.1
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Check the changes in this PR to ensure they won't cause issues with your project.


    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    cla:signed 
    opened by snyk-bot 0
  • [Snyk] Security upgrade com.fasterxml.jackson.core:jackson-databind from 2.9.5 to 2.9.10.6

    [Snyk] Security upgrade com.fasterxml.jackson.core:jackson-databind from 2.9.5 to 2.9.10.6

    Snyk has created this PR to fix one or more vulnerable packages in the `maven` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • pom.xml

    Vulnerabilities that will be fixed

    With an upgrade:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 701/1000
    Why? Recently disclosed, Has a fix available, CVSS 8.1 | Deserialization of Untrusted Data
    SNYK-JAVA-COMFASTERXMLJACKSONCORE-1009829 | com.fasterxml.jackson.core:jackson-databind:
    2.9.5 -> 2.9.10.6
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Check the changes in this PR to ensure they won't cause issues with your project.


    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    cla:signed 
    opened by snyk-bot 0
  • [Snyk] Fix for 1 vulnerabilities

    [Snyk] Fix for 1 vulnerabilities

    Snyk has created this PR to fix one or more vulnerable packages in the `maven` dependencies of this project.

    Changes included in this PR

    • Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
      • pom.xml
      • ArgusCore/pom.xml

    Vulnerabilities that will be fixed

    With an upgrade:

    Severity | Priority Score (*) | Issue | Upgrade | Breaking Change | Exploit Maturity :-------------------------:|-------------------------|:-------------------------|:-------------------------|:-------------------------|:------------------------- high severity | 701/1000
    Why? Recently disclosed, Has a fix available, CVSS 8.1 | Deserialization of Untrusted Data
    SNYK-JAVA-COMFASTERXMLJACKSONCORE-1009829 | com.fasterxml.jackson.core:jackson-databind:
    2.9.5 -> 2.9.10.6
    org.apache.kafka:kafka_2.11:
    2.1.1 -> 2.2.2
    | No | No Known Exploit

    (*) Note that the real score may have changed since the PR was raised.

    Check the changes in this PR to ensure they won't cause issues with your project.


    Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

    For more information: 🧐 View latest project report

    🛠 Adjust project settings

    📚 Read more about Snyk's upgrade and patch logic

    cla:signed 
    opened by snyk-bot 0
Owner
Salesforce
A variety of vendor agnostic projects which power Salesforce
Salesforce
The Most Advanced Time Series Platform

Warp 10 Platform Introduction Warp 10 is an Open Source Geo Time Series Platform designed to handle data coming from sensors, monitoring systems and t

SenX 322 Dec 29, 2022
Realtime SOS Android Application. Location (GPS + Cellular Network) tracing application by alerting guardians of the User.

WomenSaftey Women Safety Android Application: Realtime SOS Android Application. Designed a Location (GPS + Cellular Network) tracing application by al

jatin kasera 6 Nov 19, 2022
Time Series Metrics Engine based on Cassandra

Hawkular Metrics, a storage engine for metric data About Hawkular Metrics is the metric data storage engine part of Hawkular community. It relies on A

Hawkular 230 Dec 9, 2022
The Heroic Time Series Database

DEPRECATION NOTICE This repo is no longer actively maintained. While it should continue to work and there are no major known bugs, we will not be impr

Spotify 842 Dec 20, 2022
IoTDB (Internet of Things Database) is a data management system for time series data

English | 中文 IoTDB Overview IoTDB (Internet of Things Database) is a data management system for time series data, which can provide users specific ser

The Apache Software Foundation 3k Jan 1, 2023
Fast scalable time series database

KairosDB is a fast distributed scalable time series database written on top of Cassandra. Documentation Documentation is found here. Frequently Asked

null 1.7k Dec 17, 2022
A scalable, distributed Time Series Database.

___ _____ ____ ____ ____ / _ \ _ __ ___ _ _|_ _/ ___|| _ \| __ ) | | | | '_ \ / _ \ '_ \| | \___ \| | | | _ \

OpenTSDB 4.8k Dec 26, 2022
An open source SQL database designed to process time series data, faster

English | 简体中文 | العربية QuestDB QuestDB is a high-performance, open-source SQL database for applications in financial services, IoT, machine learning

QuestDB 9.9k Jan 1, 2023
Accumulo backed time series database

Timely is a time series database application that provides secure access to time series data. Timely is written in Java and designed to work with Apac

National Security Agency 367 Oct 11, 2022
Scalable Time Series Data Analytics

Time Series Data Analytics Working with time series is difficult due to the high dimensionality of the data, erroneous or extraneous data, and large d

Patrick Schäfer 286 Dec 7, 2022
🔥 强大的动态线程池,附带监控线程池功能(没有依赖任何中间件)。Powerful dynamic thread pool, does not rely on any middleware, with monitoring thread pool function.

ThreadPool, so easy. 动态线程池监控,主意来源于美团技术公众号 点击查看美团线程池文章 看了文章后深受感触,再加上最近线上线程池的不可控以及不可逆等问题,想做出一个兼容性、功能性、易上手等特性集于一身的的开源项目。目标还是要有的,虽然过程可能会艰辛 目前这个项目是由作者独立开发,

龙台 3.4k Jan 3, 2023
🔥 强大的动态线程池,附带监控线程池功能(没有依赖任何中间件)。Powerful dynamic thread pool, does not rely on any middleware, with monitoring thread pool function.

?? 动态线程池系统,包含 Server 端及 SpringBoot Client 端需引入的 Starter. 动态线程池监控,主意来源于美团技术公众号 点击查看美团线程池文章 看了文章后深受感触,再加上最近线上线程池的不可控以及不可逆等问题,想做出一个 兼容性、功能性、易上手等特性 集于一身的的

龙台 3.4k Jan 3, 2023
CrateDB is a distributed SQL database that makes it simple to store and analyze massive amounts of machine data in real-time.

About CrateDB is a distributed SQL database that makes it simple to store and analyze massive amounts of machine data in real-time. CrateDB offers the

Crate.io 3.6k Jan 2, 2023
Apache Druid: a high performance real-time analytics database.

Website | Documentation | Developer Mailing List | User Mailing List | Slack | Twitter | Download Apache Druid Druid is a high performance real-time a

The Apache Software Foundation 12.3k Jan 1, 2023
HurricaneDB a real-time distributed OLAP engine, powered by Apache Pinot

HurricaneDB is a real-time distributed OLAP datastore, built to deliver scalable real-time analytics with low latency. It can ingest from batch data sources (such as Hadoop HDFS, Amazon S3, Azure ADLS, Google Cloud Storage) as well as stream data sources (such as Apache Kafka).

GuinsooLab 4 Dec 28, 2022
eXist Native XML Database and Application Platform

eXist-db Native XML Database eXist-db is a high-performance open source native XML database—a NoSQL document database and application platform built e

eXist-db.org 363 Dec 30, 2022
MapDB provides concurrent Maps, Sets and Queues backed by disk storage or off-heap-memory. It is a fast and easy to use embedded Java database engine.

MapDB: database engine MapDB combines embedded database engine and Java collections. It is free under Apache 2 license. MapDB is flexible and can be u

Jan Kotek 4.6k Dec 30, 2022