ScalingSearch Main Concepts

Scaling Search Documentation

ScalingSearch is a search engine software platform that operates as a document data store using schema-free JSON data format. 

The software can run on a single hardware or virtual machine server as a ScalingSearch server instance, storing all data and indexes locally on the host server.  ScalingSearch functionality from applications is accessible through REST-based JS/SQL API calls.  The software works as a data store and search engine server blended into one single software stack.

ScalingSearch also supports highly scalable architecture when using the same software on a fleet of commodity hardware servers.  The software can be configured for cloud-like operations, where all data is distributed among hardware nodes (locally hosting smaller data shards and their replicas), and where each hardware node is running the same ScalingSearch server software.  Virtual machines can be used instead of hardware nodes.

In ScalingSearch we use following data storage hierarchy:    Document > Collection > Database

Document is the basic unit of data in ScalingSearch database. It represents a self-contained unit of data, comprised of hierarchically organized fields that are stored as a together and often queried together. You can think of a Document as object or instance of a class you operate by your application or service. Documents can be stored in JSON format. Documents are stored in Collections.

Documents are organized and stored in Collections, where each Collection stores objects of the same type or structure. We recommend to combine related Collections under one Database. Concept of the Collection is similar to a concept of table in relational database, but unlike SQL database ScalingSearch does not enforce a strict data schema.

Database consists of Collections and visually represents related collections.

Query language: JS/SQL

Documents are accessed using JS/SQL (JavaScript/SQL) statements executed against ScalingSearch. JS/SQL is based on the idea that instead of defining a proprietary query language we access objects using just the JavaScript, which is organized in a SQL-like structure.

Thnink about JS/SQL query language as SQL statements with arbitrary JavaScript embedded into them. This allows powerful, extensible transformations done on the data right in the database.

Data Model

At data insertion time ScalingSearch discovers JSON data types and creates indexes that make access to data efficient.  Customer can also at query time JavaScript code accesses documents as if they were objects in the JavaScript memory, however ScalingSearch makes sure that accessing specific fields works with indexes. 

Thus ScalingSearch offers coding flexibility, but minimizes IO requirements and computing resources necessary to fulfill the requests.