Millersoft Ltd

Apache Druid

Apache Druid is a column-oriented, open-source, distributed data store written in Java. Druid is designed to quickly ingest massive quantities of event data, and provide low-latency queries on top of the data. We support cloud and on-prem versions with Apache Superset for analysis and Apache Kafka for ingestion.

Features

  • Superset for reports, dashboards, alerts & emails
  • Turnilo for ad-hoc analytics
  • JDBC & Rest API access
  • Up to 100x faster than traditional solutions
  • Deploy in AWS/GCP/Azure, hybrid clouds, Kubernetes, and bare metal
  • Flexible security options for access and encryption
  • Tuning of ingestion and sizing of cluster for query performance
  • Drill down to line level, no aggregations required
  • Cheap storage
  • Configurable compute

Benefits

  • A modern cloud-native, stream-native, analytics database
  • Clickstream analytics (web and mobile analytics)
  • Risk/fraud analysis
  • Network telemetry analytics (network performance monitoring)
  • Supply chain analytics (manufacturing metrics)
  • Business intelligence / OLAP at massive scale
  • A modern cloud-native, stream-native, analytics database
  • Easy integration with your existing data pipelines
  • Up to 100x faster than traditional solutions
  • Unlock workflows for clickstream, supply-chain, telemetry, digital-marketing

Pricing

£250,000 a unit a year

Service documents

Request an accessible format
If you use assistive technology (such as a screen reader) and need versions of these documents in a more accessible format, email the supplier at gerry@millersoftltd.com. Tell them what format you need. It will help if you say what assistive technology you use.

Framework

G-Cloud 12

Service ID

1 1 3 4 9 6 0 6 0 8 2 5 6 1 7

Contact

Millersoft Ltd Gerry Conaghan
Telephone: 0131 376 7114
Email: gerry@millersoftltd.com

Service scope

Software add-on or extension
No
Cloud deployment model
Public cloud
Service constraints
Druid is a write only database for key facts but dimension hierarchies can be configured dynamically out with the Druid application and brought in for analysis via lookups or joins.
System requirements
Linux only

User support

Email or online ticketing support
Email or online ticketing
Support response times
Depends on SLA, normally within 4 hours
User can manage status and priority of support tickets
Yes
Online ticketing support accessibility
None or don’t know
Phone support
Yes
Phone support availability
9 to 5 (UK time), Monday to Friday
Web chat support
No
Onsite support
Yes, at extra cost
Support levels
L1: Tier/Level 1(T1/L1)
Initial support level responsible for basic customer issues. Gathering formation to
determine the issue by analysing the symptoms and figuring out the underlying problem.
L2: Tier/Level 2(T2/L2)
This is a more in-depth technical support level than Tier I containing experienced and more
knowledgeable personnel on a particular product or service.
L3 Tier/Level 3(T3/L3)
Individuals are experts in their fields and are responsible for not only assisting both Tier I and
Tier II personnel, but with the research and development of solutions to new or unknown
issues.
Severity Definitions
1- Critical: Proven Error of the Product in a production environment. The Product Software
is unusable, resulting in a critical impact on the operation. No workaround is available.
2- Serious: The Product will operate but due to an Error, its operation is severely restricted.
No workaround is available.
3- Moderate: The Product will operate with limitations due to an Error that is not critical to
the overall operation. For example, a workaround forces a user and/or a systems
operator to use a time consuming procedure to operate the system; or removes a nonessential
feature.
4- Due to an Error, the Product can be used with only slight inconvenience.
Support available to third parties
Yes

Onboarding and offboarding

Getting started
Documentation and Training Videos https://druid.apache.org/docs/latest/design/index.html
Service documentation
Yes
Documentation formats
  • HTML
  • PDF
End-of-contract data extraction
All data resides inside the customers cloud/onprem account.
End-of-contract process
Included;
Core Druid consultancy
Druid installation and configuration
Druid ingestion specifications
Druid tuning
Druid testing
Kafka installation and configuration
Supset installation and configuration
Security installation and configuration

Excluded;
Druid support
Druid upgrades

Using the service

Web browser interface
Yes
Supported browsers
  • Internet Explorer 10
  • Internet Explorer 11
  • Microsoft Edge
  • Firefox
  • Chrome
  • Safari 9+
  • Opera
Application to install
No
Designed for use on mobile devices
No
Service interface
No
API
Yes
What users can and can't do using the API
Both Druid and Superset are shipped with a comprehensive API
API documentation
Yes
API documentation formats
HTML
API sandbox or test environment
Yes
Customisation available
Yes
Description of customisation
We can accommodate and support custom configuration requests.
Tokenisation of PII information is possible.
User access, roles and authentication are fully configurable.
Different ingestion specifications and cube configurations are supported

Scaling

Independence of resources
Supports AWS Athena for serverless query access via SQL at scale.

Analytics

Service usage metrics
Yes
Metrics types
Druid generates metrics related to queries, ingestion, and coordination.

Metrics are emitted as JSON objects to a runtime log file or over HTTP (to a service such as Apache Kafka). Metric emission is disabled by default.

All Druid metrics share a common set of fields:

timestamp - the time the metric was created
metric - the name of the metric
service - the service name that emitted the metric
host - the host name that emitted the metric
value - some numeric value associated with the metric
Metrics may have additional dimensions beyond those listed above.
Reporting types
  • Real-time dashboards
  • Reports on request

Resellers

Supplier type
Not a reseller

Staff security

Staff security clearance
Other security clearance
Government security clearance
Up to Developed Vetting (DV)

Asset protection

Knowledge of data storage and processing locations
Yes
Data storage and processing locations
  • United Kingdom
  • European Economic Area (EEA)
  • EU-US Privacy Shield agreement locations
User control over data storage and processing locations
Yes
Datacentre security standards
Complies with a recognised standard (for example CSA CCM version 3.0)
Penetration testing frequency
Never
Protecting data at rest
  • Physical access control, complying with CSA CCM v3.0
  • Physical access control, complying with SSAE-16 / ISAE 3402
  • Encryption of all physical media
  • Scale, obfuscating techniques, or data storage sharding
Data sanitisation process
Yes
Data sanitisation type
Deleted data can’t be directly accessed
Equipment disposal approach
Complying with a recognised standard, for example CSA CCM v.30, CAS (Sanitisation) or ISO/IEC 27001

Data importing and exporting

Data export approach
Various export utils via command line and JDBC
Data export formats
  • CSV
  • Other
Other data export formats
Json
Data import formats
  • CSV
  • Other
Other data import formats
  • Kafka
  • AWS S3
  • Networked Drive

Data-in-transit protection

Data protection between buyer and supplier networks
  • Legacy SSL and TLS (under version 1.2)
  • Other
Other protection between networks
Can also encrypt prior to transfer
Data protection within supplier network
TLS (version 1.2 or above)

Availability and resilience

Guaranteed availability
Customer dependent.
Approach to resilience
AWS services are delivered from multiple datacentres worldwide. When deploying customer services to AWS, DataNessie can be configured such that services span multiple availability zones (data centres) to ensure service resilience. Alternatively, our Disaster Recovery as a Service offer can be used to provide DR.
Outage reporting
AWS Cloudwatch alerts can be created

Identity and authentication

User authentication needed
Yes
User authentication
Username or password
Access restrictions in management interfaces and support channels
Access to management interfaces and support channels is restricted through a combination of username and passwords, multifactor authentication, firewalling, IP restrictions, the use of bastion hosts as appropriate.
Access restriction testing frequency
At least once a year
Management access authentication
  • 2-factor authentication
  • Public key authentication (including by TLS client certificate)
  • Username or password

Audit information for users

Access to user activity audit information
Users have access to real-time audit information
How long user audit data is stored for
User-defined
Access to supplier activity audit information
Users have access to real-time audit information
How long supplier audit data is stored for
User-defined
How long system logs are stored for
User-defined

Standards and certifications

ISO/IEC 27001 certification
No
ISO 28000:2007 certification
No
CSA STAR certification
No
PCI certification
No
Other security certifications
No

Security governance

Named board-level person responsible for service security
Yes
Security governance certified
No
Security governance approach
Dependent on configuration of each installation
Information security policies and processes
Depends on the specifics of the installation, if on AWS then we follow https://aws.amazon.com/security/

Operational security

Configuration and change management standard
Supplier-defined controls
Configuration and change management approach
All code is under version control using git
Jenkins is used to build releases
An automated test framework is used for integration testing
Changes are tracked via jira
Cloudformation is used to deploy via AWS Marketplace
Vulnerability management type
Undisclosed
Vulnerability management approach
Solution is deployed into customer's AWS VPC via AWS Cloudformation
External access is configured via customer and GUI is locked down via AWS security groups
SSH access is also locked down via security group and PEM file.
The access is as secure as the customers network.
Patches are in the form of new AWS AMIs
Protective monitoring type
Supplier-defined controls
Protective monitoring approach
All logs go to AWS Cloudwatch for auditing, monitoring and alerting
Incident management type
Supplier-defined controls
Incident management approach
Each Data Nessie instance runs within a VPC within the customers AWS Account. There is no external access or monitoring. Issues need to be reported to the supplier and logs supplied for external analysis.

Secure development

Approach to secure software development best practice
Supplier-defined process

Public sector networks

Connection to public sector networks
No

Pricing

Price
£250,000 a unit a year
Discount for educational organisations
No
Free trial available
No

Service documents

Request an accessible format
If you use assistive technology (such as a screen reader) and need versions of these documents in a more accessible format, email the supplier at gerry@millersoftltd.com. Tell them what format you need. It will help if you say what assistive technology you use.