SingleStore will measure the “data intensity” of your application – Blocks and Files

SingleStore introduces a “Data Intensity Index” to help companies understand how data-intensive their RDBMS applications are and how SingleStore’s database can cope with high levels of data intensity.

The company provides a relational SQL database that can handle transactional and analytical workloads, and can run on-premises or in the public cloud. Akamai, Comcast, Royal Bank of Canada and Uber are customers. When a customer requests a ride during peak times, like New Year’s Eve, Uber presents a real-time peak price within milliseconds using SingleStore.

A blog post by SingleStore’s Principal Solutions Engineer, Ian Gershuny, discusses the company’s data-intensive concept and states, “The truth is that many of your existing applications should be data-intensive,” but this is not the case, because “we limit our designs to bottlenecks”. We don’t bring in too much data so our loading process doesn’t choke. We report on outdated data as our processes are consolidated into our analytics and reporting databases. We limit access so as not to overload the database.

Gershuny says, “Even if you are able to circumvent these limitations, our need for data-intensive applications is in our future.” An example is delivery trucking: “Five years ago, it would have been inconceivable to follow a UPS truck through a neighborhood. Now I watch it directly on my phone.

This data-intensity concept is a clever marketing way to encapsulate SingleStore’s message; its combined transaction and analysis architecture can cope with greater data intensity than separate databases. Data Intensity Index is a SingleStore notion for an index value based on five variables: data size for an application, ingestion speed, query complexity, query latency (time to execution) and concurrency (number of users).

Businesses can enter their app’s values ​​for variables in a SingleStore website page by answering 10 questions:

  • Data set size (1 TB, 1-10 TB, 10-50 TB, 50-100 TB, > 100 TB)
  • Data growth rate (100%)
  • Data ingestion rate in rows/sec (1m)
  • Complexity of queries in joins (1-2, 3-5, >6)
  • Required query execution time (minutes, 1-10 seconds, 100ms-1second, 10-100ms, 0-10ms)
  • Number of users (100,000)

The assessment is supposed to take three minutes and results in an index score between zero and 100. The higher the score, the more data-intensive your app is, but the score boundary between a high-intensity app and a lower intensity application those or even low intensity ones were not provided by SingleStore. All is relative.

The report also provides an assessment of the type of data infrastructure that SingleStore estimates an application will need to provide the best user experience.

Read Gershuny’s blog for an overview of the SingleStore concept and see if it resonates with you. A downloadable white paper is available here.

Lance B. Holton