Versatile Clustering Options

One of the most cost-effective data processing approaches today is to use a scalable database technology at infrastructure level to run large cluster databases using expendable commodity hardware networked into clusters.   

Easy clustering using standard off-the-self hardware equipment can provide for both quick resolution of hardware-related technical problems and for flexible performance and storage capacity increase when necessary, without paying for costly high-end solutions.  

Application workload also can be shared across a number of many servers to accommodate transactions from many simultaneous users typical of modern web application environment.

It is popular to call this IT architecture pioneered by some large Internet services a "cloud" computing or cloud-architecture. 

Clusterpoint Server database platform software was designed as a generic distributed database software, which operates as transparent cluster database management system.  It fully supports cloud-architecture principles, and can be used to build customer private cloud database infrastructure.

Clusterpoint DBMS supports all three major scale-out options illustrated in the picture on the right.

Combining those three basic clustering options as necessary, our customers can address virtually any required database scalability and availability level for their corporate databases and related applications.  Please read below more details about each of the specific clustering options.

Classic Distributed Database

Basic clustering option supported by Clusterpoint technology is a classic database partitioning into multiple parts (shards) which together form a single logical database from application developer point of view and from database administrator (DBA) point of view. 

In this configuration any database can be split among as many cluster nodes as necessary, to scale overall database data storage capacity and performance.  Each cluster node will run only its own data storage containing 1/Nth of all N database XML documents.  That is the reason we call this cluster configuration a database striping (from a well-known analogy of a RAID disk storage principles).  

All clustering functionality will be performed by Clusterpoint Server software, absolutely transparently from Clusterpoint API users.  Actually application developers may even not know about number of underlying hardware cluster nodes, handling the total database volume.  This is configured only by an DBA or system administrator, using web based Clusterpoint Manager utility.

Administrator can flexibly add new cluster nodes to already striped database cluster in Clusterpoint architecture, without taking down or reorganizing existing database cluster configuration.  It allows to increase database storage capacity in elastic way, without services downtimes.

Clusterpoint database striping provides classic advantages of application and data storage scalability across a distributed database architecture in a cluster of networked hardware servers.  The key advantages of Clusterpoint distributed database support are:

Database size scalability:  match increasing your database size and data volume storage capacity with inexpensive commodity hardware, adding it to the striped database configuration as necessary;

Search performance scalability: Clusterpoint Server will automatically partition your database into N smaller parts if striped database configuration is used for any Clusterpoint database storage; by increasing of number of cluster nodes, you effectively decrease the total size of the database storage part (shard) managed by each hardware server, and thus increase overall search performance of the total database system;  Clusterpoint Server automatically performs search query distribution and execution on all cluster nodes, resulting in fast sub-second search into complete database, using specially engineered Clusterpoint Index and Clusterpoint Information Ranking for search relevance;

Indexing performance scalability:  by partitioning the total database in 1/Nth of content, to be stored, managed and indexed per each cluster nodes, you can effectively match your database indexing performance needs to your business requirements;  if your application demands minimum downtime, choose more number of servers in a cluster, to linearly decrease indexing time necessary for short indexing times even during massive data uploads or re indexing procedures;  

Automatic indexing workload sharing: Clusterpoint Server software automatically determines which cluster nodes should be selected to store incoming database updates, based on combined load sharing CPU workload estimate and number of documents per each cluster node database storage

Customizable application indexing logic:  you can freely customize Clusterpoint indexing and request that you documents be indexed by a specific cluster node only; it is useful if you need to partition indexing logic into your application software, organizing your database content on different cluster nodes by some higher level criteria (for example, by geography, language, by application, by time period etc.).  There is Clusterpoint API parameter enabling you to always address specific cluster node, if you want to avoid some cluster-wide functionality implemented on the Clusterpoint Server;

Resilience against equipment failures:  failure of any node will not result into loss of complete database availability making databases services totally unavailable; only 1/Nth of striped database documents will be temporarily unavailable for search, while malfunctioning hardware will be replaced; all other parts of the striped database will continue to function run by other cluster nodes;

Centralized cluster-wide management: You can set up and configure a striped database storage (that we simply call a 'cluster storage' in our database platform architecture) from our web based Clusterpoint Manager utility.  You can also manage and control any cluster storage from this centralized utility with a one-click status or configuration changes: run / stop/ re index etc.  All administration commands by default will be executed for all cluster nodes, cluster-wide, without the need to address and configure specific cluster nodes. 

Please note that Clusterpoint Server software comes as well-supported enterprise software for your XML-only database management.  Fully controlling and developing source code of Clusterpoint Server and its management utilities, we can quickly add some custom clustering option or extra management feature to the core Clusterpoint database platform software, required by our customers.  Please do not hesitate to contact us by letting know what may be valuable, just sending us email on Clusterpoint Support Email. 

Database Mirroring into Many Copies

Clusterpoint Server software has been designed to supports a full database mirroring into multiple working copies, running on different hardware nodes in a cluster.

Using Clusterpoint Manager, DBA can create and configure multiple database storages with the same name, located on different hardware nodes, and configure them to ran as a single databases from application developer point of view.

It can be done at database configuration level, using Clusterpoint Manager, with setting up a required number of fully redundant database mirror copies in controlled cluster locations and always precisely knowing where your database mirror copies reside and work.  This is another great advantage of Clusterpoint Server database software platform, in particular, comparing with key-value data store software, that can not guarantee where redundant data is kept and thus may cause problems during emergency data recovery where this knowledge is essential.

The full database mirroring is being done automatically by Clusterpoint Server software, including synchronization of updates and deletes, and there is no need to implement this clustering logic into customer application software functionality.   There are also Clusterpoint management tools to synchronize replicated database mirror content by temporary setting some cluster node as a master, if necessary.

Actually application may even do not know if there  is only a single copy or multiple copies of the same identical content database running.

Here are advantages of this our approach to mirror and run multiple copies of the same database in a cluster:

• Automatic on-line backup for a full database:  frequently people to forget to do scheduled database backups properly, or it is rather tedious and mundane task for large databases performed in off-hours or weekends;  With disk space and hardware costing almost nothing today, we decided to build the database software, which performs all the backup functionality on-the-fly, on another hardware, by mirroring the full operational database among as many backup copies as necessary;
• Workload sharing:  using Clusterpoint's built-in load-balancing tools, or even better, at an application server or at a web server level (to avoid any single-point of failure), you can scale your search load balancing among as many cluster nodes as necessary, to match your number of users and performance requirements;
• Resilience against hardware failure: failure of any node will not affect application software functionality making databases services totally unavailable; instead only performance will be affected, as all transactions can continue to be executed on other identical cluster nodes;
• Automatic fail over:  when combined use with load sharing tools, Clusterpoint Database, fully mirrored into multiple working copies, will automatically switch to another node (or mirror) to perform database transaction; while malfunctioning equipment will be replaced;
• Automatic or manual synchronization of updates:  whenever any database node is updated, its all mirrored backup copies are updated in synchronized way; if any node is malfunctioning, it can be taken down using Clusterpoint Manager, and database synchronization can be automatically restored, when hardware problems are fixed;  DBAs can also manually force to re-synchronize a database copy against any other node, taken as master for that task;
• Centralized cluster-wide management: You can set up and configure a mirrored database storage (that we simply call a 'mirrored storages' in our database platform architecture) from our web based Clusterpoint Manager utility.  You can also manage and control any cluster storage from this centralized utility with a one-click status or configuration changes: run / stop/ re index etc.  All administration commands by default will be executed for all mirrored database cluster nodes, cluster-wide, without the need to address and configure specific database mirror nodes for changes to take effect.  It is a time saver for DBAs. 

This full database mirroring (replication and synchronization) functionality usually is available for legacy commercial database software only for extra cost, in many cases requiring also to address those issues in application software, configuring the database synchronization logic in specific custom application way.  Clusterpoint Server database platform delivers this functionality out-of-the-box. 

Multi-datacenter Cluster Replication

Clusterpoint Server software also supports mirroring of complete distributed database (a striped database) into multiple copies.

This feature may be useful for those customers who would like to provide the highest possible level of scalability and availability for their mission-critical services.  For example, the customer can have multiple data centers where he wants to run some very large cluster database, completely replicated (mirrored) for security and high-availability reasons into another data center.

In Clusterpoint architecture our customers can easily establish as many mirror copies of a cluster database as necessary.

Technically each cluster node handling 1/Nth of the database and configured to run into striped configuration with other nodes, can be mirrored to other hardware nodes over the network, creating a node-based "small" sub-mirror for that particular part of the total database.  All database update transactions will be synchronized only among sub-mirror nodes, using Clusterpoint Server built-in database mirroring facility, resulting in running in parallel N sub-mirrors, each servicing multiple copies of each clustered database node.  

Key advantages of this combined clustering approach beyond those described for the striped and the mirrored database configurations are following:

• Very high level of availability: if your applications require 99,99% availability, the best clustering method;
• Fail-safe system: equipment failures would not affect database performance;
• Fully redundant use of infrastructure:  when run in different data centers, services will continue to operate even if one of data center becomes unavailable or malfunctioning (access problems like earthquakes, lost Internet connectivity, loss of regional power supply etc.)

Please note that building of geographically distributed database infrastructure will also require fast network links, as all database updates must be carried out in synchronized way.  Large number of cluster nodes can demand high through output links, if the cluster database is mirrored to other data centers.

Please also note, that for geographically distributed database infrastructure you would need to set up fast VPNs secured by encryption and adjust firewall configuration to provide inter networking protocols for Clusterpoint Server software and its centralized management system, for example, for automatic discovery of new hardware nodes, installed with Clusterpoint software (please see also Clusterpoint Management).

From other aspects of cluster database mirroring, the Clusterpoint Server software does not require specific custom software versions to provide this level of database services availability.  The very architecture and design of Clusterpoint's XML-only database platform provides it out-of-the-box.  In our architecture all core functionality is scalable:  Clusterpoint Indexing based on Clusterpoint Information Ranking, Clusterpoint Search and Clustering. 

Cost-Efficiency of Cluster Storage

Whenever it is necessary to scale legacy SQL database architecture, it usually requires more and more computing resources: powerful equipment, specialized software licensing, extra maintenance efforts etc.  Costs of equipment, software and related running costs start to escalate.

By distributing computing tasks on simple and standard infrastructure components the cost of maintaining and operating the total system is fixed per user and per gigabyte of data:

The costs do not escalate when system capacity needs to be increased.  The total cost of system ownership can be planned using a fixed costs model (per user, or per gigabyte), representing an attractive return on investment.

This approach has been proved by some of the world's largest Internet companies providing Internet services to tens and hundreds of millions of users.

With Clusterpoint Server database platform software you can start building your own scalable private-cloud architecture database storage infrastructure for your corporate‹ databases and applications, achieving economies of scale, database search performance and end-user search relevancy, that this our software provides.  

Please read also Customer Advantages.