Many people still have some doubts that NoSQL databases can perform as well as legacy SQL relational databases in scalability and performance. To demonstrate Clusterpoint Server database software features, in particular, performance and scalability, we are developing, maintaining and supporting a portfolio of vertical applications, which are called Clusterpoint software application products.
They are all based on Clusterpoint Server functionality, where Clusterpoint DBMS software is being either included into the packaged application, or must be licensed separately to serve as a database storage and search facility.
Each application targets some usability are of enterprise information management, and can be used for solving real customer problems. In particular applications are addressing problems of IT business sectors where a lot of totally different by structure information needs to be stored and managed in massive data sets naturally requiring database clustering for the most cost-effective solution, at the same time maintaining high performance at database search and access. In this way - through vertical applications, where Clusterpoint Server software works beneath the application layer, invisibly to our customers, we can prove usability of Clusterpoint DBMS platform across various business cases.
There is the following Clusterpoint software application products available from our company:
Clusterpoint applications, except Crawler and Searcher, each has their own separate commercial licenses, and can not be used under Clusterpoint DBMS Free Community License, Evaluation License or other DBMS-only licenses.
Although each application is being driven by a fully-functional Clusterpoint DBMS, there are different licensing models due to the industry specific practice, usually depending on a specific customer network setup, hardware availability and performance requirements.
Below each application product is described in more details.
Clusterpoint
Network Traffic Surveillance System Application - complete
application, storage and search system for
capture,
analysis, reconstruction and permanent retention of all network traffic
traversing corporate network boundary with Internet, or a particular
corporate network segments of interest.
Application captures and reconstructs all raw IP
traffic from packets to OSI level7 application protocols,
performs
storage of
all captured traffic objects into scalable and entirely searchable
Clusterpoint DBMS, and performs full content indexing for all objects
for instant search (including all Web pages, web posting,
emails,
video, audio,
images, chat, ftp, documents transferred, ssl-traffic connections, dns,
udp, ftp data transfers etc.), along meta data such as user data, IP
address, source and destination ports, type of traffic, direction of
traffic, etc.
The Clusterpoint NTSS application system is technically
engineered to work as a custom-build
high-speed application software developed in C/C++ that
"listens"
in stealth or bridge mode on any network link, intercepts all raw TCP
and UDP traffic IP data packets, reconstructs the traffic back
to Application protocol (OSI Level 7) levels, extracts from resulting
connections all data objects such as Web pages, images,
videos, audio,
web postings, emails, documents attached or transfered over the
network, ssl-traffic connections, dns, udp, ftp data transfers, chat
messages etc., and indexes all that relevant traffic along meta data
such as user data, IP
address, type of traffic, direction of
traffic, etc. into
Clusterpoint scalable DBMS at high speed, for instant access, full
content search and retrieval. NTSS application indexes
absolutely different in size and structure data objects into a single,
entirely searchable database of all network traffic crossing the
corporate network boundary with the Internet. It
also adds network related meta markup such as source and
destination addresses and ports, relevant timestamps, domain names,
user names, types of traffic etc., to provide cross-search and data
pivoting in different navigation views.
Unlike proxy servers, Clusterpoint NTSS:
Clusterpoint NTSS stores also network traffic content and delivers lightning-fast sub-second search in the content of that traffic. With Clusterpoint NTSS all network traffic can now be automatically captured, stored and indexed into Clusterpoint XML database, and our customers are empowered to perform at any time instantly responsive search into the actual content of all that traffic, by simply querying the content of stored traffic using Internet-style ad hoc querying supported by underlying Clusterpoint DBMS. NTSS is a game changing software application, simplifying IT infrastructure for corporate long-term data preservation: it stores everything, it scales out in private cloud configuration, and is instantly searchable.

The
Clusterpoint NTSS application is extraordinary easy to use for any
authorized personal for basic security analysis of cyber security
incidents. It does not require deep technical knowledge about
network
protocols, log file formats or specific corporate application
details. It
empowers security personnel with basic IT knowledge to do analysis for
any past network
activity over the corporate network boundary using easy-to-use Web
interface, even to replay past traffic as it was seen on a user PCs in
particular time frame by particular user step-by-step, surveilling the
past traffic and actions taken by user. Using simple
Internet-style ad hoc search dashboard (please read more about Clusterpoint Search),
security personal can quickly, without knowing deep technical
details of network and protocols, search and access any stored network
traffic in full entirety, including content of documents, Web pages
viewed by employees, videos streamed long time ago etc. Here
are few sample screen shots from Clusterpoint NTSS:

Please
note, that the Web page you see in the picture above, is shown from the
Clusterpoint DBMS database, reconstructed from raw IP traffic,
collected somewhere in the past and stored in Clusterpoint NTSS data
storage. All Web page elements are restored fully, showing
the
actual screen that user requesting that web page, did view on his PC at
that particular time-frame.
You
can search past network traffic also by meta data, for example, by
user, in specific
time frame, from specific address, filtering out only relevant traffic,
and chronologically replay the activity of users on the Internet,
viewing and analyzing traffic exactly as they have seen it on
their computer screens or transacted over email or chat in any moment
of the past. Even NTSS startup interface is
absolutely simple -
just a single search box you can instantly start your traffic
inspection and analysis job for a suspected particular security
incident where you know at least something: a user name, time frame,
IP,
port number, a time stamp, session code, cookie content of something
else. Just enter few simple search keywords or strings, and
the system will instantly find out in what context, in what type of
traffic, or by what users this query matches their traffic, and enable
you to go directly to that particular time frame and IP address, to
check the whole sequence of network transactions of past traffic
activities for eventual cyber-incidents proof of evidence, replaying
the
web traffic
as it was seen on users computer, or copying out from Clusterpoint
NTSS database all evidence for forensic needs such as emails,
documents, Web pages, or even raw IP and UDP traffic data packets.
It can also perform
other useful
functionality for resolving cyber security incidents: has a built in
statistics module to identify suspicious activity on the network,
alerting filters to automatically send notices about potential security
breaches if certain content is detected in web pages, emails or
documents transfered over the network, advanced search and reporting
facility enabling to perform regular monitoring of required events etc.
Like the Clusterpoint database software itself, NTSS application software is designed to take advantage of Clusterpoint DBMS generic cluster architecture (please read more about Clusterpoint DBMS cluster database functionality in section Clustering). Multiple NTSS-software installed servers on corporate network boundary (monitoring multiple links) can capture, reconstruct and establish a single logical, physically distributed, Clusterpoint DBMS storage containing huge amounts of network traffic. All the network traffic can be stored permanently for weeks or even months, if disk subsystem storage space allows.

All
the stored by NTSS traffic is instantly searchable for security
analysis and forensics using very simple NTSS web UI , making an NTSS
quite a useful tool for resolving various cyber-security incidents,
requiring access to full traffic content, up to raw IP and UDP packet
data level, which are also permanently stored along re-constructed
application level
objects.
One can say that NTSS operates as a sort of time-machine,
you can use it to turn back and "replay" the network traffic of
interest at
particular point of time, reviewing past network incidents data fully
and user-friendly.
We use Clusterpoint Information
Ranking to deliver high speed full content search in network
traffic, which creates pre-sorted indexes
and thus does not slow down performance of NTSS user interface and
search functionality even when used for analysis and search in very
large databases containing all corporate network traffic for
months, and distributed for storage among many servers in cluster.
There is no other economical way how to store all that
traffic volume without clustered or so called "private cloud"
architecture database. Clusterpoint NTSS enable to
accommodate truly
massive data volumes in cluster database, and still perform
relevant search across all of data in a fraction of a second.
There is open Clusterpoint API enabling access to stored traffic objects directly from many 3rd party applications in Clusterpoint NTSS application platform.

Our API is useful for
specialized analysis, risk management, security reporting, productivity
increasing business knowledge applications and so on. All
database objects are XML-formatted, and very easy to integrate and work
with. All captured network traffic together with meta data is
stored
into Clusterpoint DBMS XML storage with its open API, which, if
permitted, exposes all
those stored objects to any other higher-level corporate applications
performing business analytics, risk analysis, security monitoring or
productivity monitoring tasks.
Most of other 3rd party software nowadays support XML.
For existing applications Clusterpoint NTSS can serve as the
most
useful infrastructure tool to collect, store and enable fast accesss
and
search to
complete network traffic backup. Then you can use your own
existing
high-level software tools and applications to process that data for
specific
business intelligence needs.
For your customers, who may be interested in Clusterpoint NTSS
system, please see also a downloadable PowerPoint
presentation in
our web site section Download
/ Demonstration Area.
With Clusterpoint NTSS everything trafficking over your
network
becomes transparent, manageable and searchable!
Read more in
the NTSS product Web site by visiting Clusterpoint NTSS.
We have also developed an advanced state-of-the-art Internet search platform application with cluster-enabled crawler written in C/C++ and designed to run on scalable cluster of servers, for automatic Web link discovery and spidering in the whole global or national Internet domain name space and IP address zones. It stores all collected data into scalable and searchable Clusterpoint DBMS XML storage. It also applies customizable ranking based algorithm for making the resulting Internet index database as meaningful for relevant search as services provided by leading Internet search vendors.
The key functionality of the GISP software platform is to build
a massive distributed among tens and hundreds of servers database and
its search index required for large scale Internet search services
venture. For example, with GISP software you can effortlessly
from the application software point of view build a
country-wide national search engine, targeting areas or innovative
business models where one-size-fits-all global search engines may
struggle
to perform due to global spam problems, local search relevance
weaknesses, or insufficient attention to specific national markets.
Please note that there are two different applications in our product
line - Global Internet Search Platform (GISP) and
Clusterpoint Crawler Application, please see below. The
main difference between both applications is that Clusterpoint Crawler
Application is targeted for corporate internal use only and designed to
crawl
manually configured internal and external company resources.
GISP is much more advanced application
than Clusterpoint Crawler Application and performs automatic Internet
link traversing,
discovering new domain names and IP addresses with Web resources to
crawl, it performs cluster database storage management, calculation of
document importance rankings across the whole cluster and constantly
updating the ranking upon each new crawling process, incremental
re-indexing of fresh information and a lot of other functionality,
which is required by a full-fledged Internet crawler.
GISP
also collects data using multiple parallel processes on multiple
servers, and Internet spidering processes are
multi-threading. It partitions the total downloaded
data into separate cluster nodes and separate indexes stored on
multiple servers. This indexing method needs to be
accommodated
to available corporate bandwidth and network topologies. GISP
also handles discovery
and elimination of duplicate content or so called alias-pages, handles
standard HTML meta tags and custom XML tags, and does a lot of other
content filtering and
adjustments to fine tune results to the highest Internet search quality
and best user experience as possible.
GISP application crawls and indexes Web page content, META data such as
Title, Meta keyword, Meta description, page
size, language, download date, text content in local and related HREF
links, domain names
etc. This meta data is stored in the Clusterpoint Server
driven database storage, together with full text index for
that data, and can be used to develop functional advanced
search functionality which is as powerful as that of the industry leaders
providing global Internet search services. However, using
Clusterpoint Information Ranking, you can apply your own information
sorting rules for relevance, based on much better knowledge of your business,
for example, targeting more deeply specific Web content and covering
twice or triple as much Internet search index data for a particular
geography than your competitors.
It
is a well known fact that leading global players in Internet search
area
crawl and index only less than 2% of Internet publicly available data
so that there is still plenty of room for new services in Internet
search market area. We at Clusterpoint does have a scalable
infrastructure technology that you can use to take competitive
advantage in this still nascent market, with nearly unlimited
possibilities for our customers to go global, for vertical search
sectors, covering specific geographies with radically better and wider
local search, or pursuing innovative business models in this growing
market.
Besides standard Web pages GISP indexes also other most popular file
formats on the Internet such as XHTML, XML, RTF, TXT, DOC, PDF, MS
Excel, MS PowerPoint, Postcript etc. GISP converts file
content to text and also stores original documents
into Clusterpoint DBMS data storage for later retrieval from GISP as
cached document originals. Saved versions of the
original files could be used by the applications to view cached version
of Web documents when the source server is not available.
Every large scale application is custom by its nature, and we deliver
GISP software solution only together with our mandatory installation
and
configuration services. For crawling the best and most
relevant data and delivering to end
users the best Internet search results, we need first to set up GISP
software in our customer data center environment, usually on a custom
hardware configuration, in specific cluster network topology (which
again is determined by the size, performance and scalability
requirements of the application and customer business needs), configure
domain names and IP address zones for crawling, together with our
customer establish the best crawling and update policy, indexing and
content download limits etc. Essentially, every delivery of our
GISP
software is customizable for our customer needs. To build a large
scale Internet search
engine: this is still as much art as it is a pure technical venture in
most
cases.
Therefore we should always assist our customer to set up the GISP
system and perform the first two full crawls of the
Internet using GISP software (the second full crawl is minimum
necessary to achieve good relevance rankings).
In most cases we can assist our customers remotely over the
Internet and support
customer organization with live technical assistance using such VOIP
software as Skype.
Compared to existing solutions based on integration of disparate search
and database products, the biggest advantage of using Global
Internet Search Platform for running
the Clusterpoint Server database for a country-wide or global Internet
search engines is less demanding
hardware resources for the same or even better performance levels and
much less complex job to set it all up, manage and operate .
Simply speaking, you will need less servers and less man-power to
operate your country-wide Internet search engines with Clusterpoint
software. Another
advantage is that GISP software does not require complex integration of
multiple products. GISP is also commercially supported
software platform by our team of experienced software engineers working
on large scale search services since 2001.
GISP
was initially developed to prove Clusterpoint
DBMS scalability and search quality at large scale projects, involving
the need to manage both structured and unstructured data.
Clusterpoint DBMS software can serve as foundation for any
similar cloud database solutions where you can build massively
large databases such as public access email or file storage services,
Wikipedia-style knowledge resources, social-networking services,
e-commerce stores, all without losing search performance and
quality when database and index grows big and needs to be distributed
in cluster architecture. Please read
more about Advantages.
In partnerships with businesses having enough spare hardware resources
and
bandwidth, GISP software can be used by to build truly large
and scalable
global Internet search engines in clusters of up to several thousands
of computers and billions of objects (web pages, documents etc.).
For those customer we can supply custom
installation of GISP software, adopted to their network hardware
topology, which needs to be set up, hard-wired and configured in a
tree-like hierarchy of hardware cluster nodes. When run in a
tree-like hard-wired network infrastructure, the same Clusterpoint
software can run on all servers and still perform sub-second
search queries without performance loss. The architecture for
Clusterpoint DBMS software has been designed with this required
scalability.
Please
note that Clusterpoint GISP system is proven by our customers in
production and it is running as the core IT software used by an
Internet service vendor providing a country-wide Internet search engine
service.
We offer free test drive of GISP software on our company demo servers
for a smaller amount of country-wide Web data, to see if the software
can be used for an Internet search
engine functionality and satisfies performance for your country or for
your
Internet specific application content.
Please contact us to ask for the GISP demo terms and availability on Clusterpoint Customer Email.
Stripped-down corporate file system and Internet crawler application written in PHP and C/C++, does not traverse Web links automatically, the list of crawled domains needs to be configured manually. Has a specialized task system for setting up tasks that regularly and automatically crawl specified corporate or Internet resources, at necessary level of details and frequency, and store data in particular Clusterpoint storages. Free, included with each Clusterpoint Server license. Together with Clusterpoint Searcher Application can be used to test drive Clusterpoint Server performance and functionality as a simplified bare bone corporate search appliance. Functionality of Clusterpoint Crawler is the following:
A PHP open source sample code for crawled Web and corporate document database search, which can be modified and installed as plug in for customers corporate CRM/CMS applications, providing web based access and search to regularly crawled by Clusterpoint Crawler application content. Free, included with each Clusterpoint Server license. Together with Clusterpoint Crawler Application can be used to test drive Clusterpoint Server performance and functionality working as a simplified bare bone corporate search appliance. It is a demonstration application for Clusterpoint Server illustrating storage of complex objects, ad hoc search and structured search capabilities in Clusterpoint storages (XML-document databases) collected by another Clusterpoint demonstration Application - Clusterpoint Crawler.
Clusterpoint Searcher application is included in each Clusterpoint Server installation free of charge; and is being provided as a customer-customizable PHP-based Web server module; it is also very simple to integrate into other corporate applications as a bare bone sample front-end code for Clusterpoint XML-/JSON- only databases containing corporate documents and Web content; Clusterpoint Searcher Application provides access, search and navigation in all databases (storages) created by our customers and filled with data by using a bundled Clusterpoint Crawler application; Clusterpoint Searcher is a BONUS application, we provide it free of charge under free software license as a reference client code, and you can modify, extend, transform this application source code as necessary for your own corporate IT environment; you can also use 3rd party software instead of Clusterpoint Crawler to collect corporate data and provide it for Clusterpoint Searcher, taking the sample XML / JSON data structure for database objects, what Clusterpoint Crawler creates and uses, and storing your corporate objects with this structure into your own database storage by your own specialized application.
The main functionality of Clusterpoint Searcher Application is the following:

Pre-packaged by Clusterpoint hardware and software solution (available also as a software-only installation over Internet) to deploy and operate in natively scalable clusters, the whole system running as a large powerful corporate search appliance. Our search appliance software advantages:
You can go to download CSA Data Sheet from the Clusterpoint Search Appliance Web Site.
You are welcome to make inquiries about CSA on email on Clusterpoint Customer Email.
We encourage application developers to design and develop their own application products that take advantage of Clusterpoint DBMS unique open cross-platform XML architecture with its unique cohesive integrated database and search engine functionality that effortlessly scales out in clusters and does not loose performance at relevant Internet-style full text search. We believe that there are many opportunities in different business sectors where fast growing data volumes demand inherently scalable applications and customers demand instant database responsiveness at information access.
There is a new 3rd Party application software coming soon from Clusterpoint Ltd. subsidiary Clusterpark Ltd.: transaction Log Analyzer Application, that enables to create, manage and search consolidated enterprise-wide log records data from multiple servers and software services. Please see section 3rd Party Products for more information.
Clusterpoint Application software products are fully supported by our team of database and software engineers. Please see section Clusterpoint Support Services for more details. Please also do not hesitate to contact us with your inquiries about technical support options and conditions, by sending us and emails on Clusterpoint Support Email.
For Clusterpoint application software product demonstration, in particular, obtaining evaluation software for Clusterpoint applications (unlike Clusterpoint DBMS software, application software is not freely downloadable), software licensing details, pricing and product ordering and payment instructions please contact us on Clusterpoint Sales Email.
There is Clusterpoint NTSS Demonstration video-cast available in our section Download / Demonstration Area.
Please also see section Store.