GemFire Enterprise Data FabricGemStone Systems Inc. Michael I Lazar, Federal Technology Director Michael.lazar@Gemstone.com 301 325 8405 Steve Rixse, Director Public Sector Steve.Rixse@Gemstone.com 301 664 8494
GemFire Real-Time Events
21 is a complex event processing (CEP) solution that enables an enterprise to facilitate agile, event-driven architectures by analyzing streams of data in conjunction with other static/historical data sources Define relevant business events through simple SQL queries Discern patterns through continuous querying and event correlation with other data sources Distribute appropriate information to selective client applications Target Environments Fast-moving data streams Real-time pattern detection and analytics
Enterprise Data Fabric - What is it ?
22 Middle tier Distributed Data Infrastructure Operational Data Combines Distributed Caching (main-memory), DB semantics, Reliable Messaging/ Intelligent Routing and Continuous Analytics Distributed Data Space Data warehouses Applications Rational databases
Middle tier high performance Distributed data management For operational data (current working set, fast moving data and not historical ) Combines Distributed caching (main-memory) + DB semantics + reliable messaging and intelligent routing, + continuous analytics (let us leave this for the moment, CQ will be explained later) Think of it as a distributed data space where all operational data required NOW by applications is available. It is a virtualized data space – i.e. the location of the data typically can be transparent. Its most important characteristic is that it is fast – primarily based on main-memory for managing data. It provides distributed caching features – managing objects and distributing these to other processes, but also includes DB semantics such as being able to index and query the data set, engage in transactions and disk persistence + recovery. Pub-sub semantics and reliable message delivery guarantees like a messaging system and finally "continuous analyti
Where is the data in the Fabric?
23 All across the Network NETWORK IS THE DATABASE Cache Cache Cache In Local process memory Copies in multiple nodes
Data is spread across the network Data maybe local to your process, to your machine, copies in multiple machines, spread across machines, on disk or even simply accessed lazily from some enterprise repository One way to think about this : Network is the Database Or it is an abstraction that combines data in many nodes or from many repositories makes it visible as it was just one logical database
Enterprise Data Fabric - Physical View
24 All across the Network NETWORK IS THE DATABASE Cache Cache Cache Rational databases Spread across many nodes On Disk Or lazily fetched from external source Clustered Application
5 Key Features
25 1. Data Storage Optimized concurrent main-memory structures Topology Embedded in application process Client caches and Cache servers Partitioned across many servers n-way in-memory replication for HA On Disk Overflow to disk Recoverable disk regions – Asynchronous or Synchronous
5 Key Features
26 2. Access Model Object Caching API (get()/put()). Querying through Xpath, OQL and SQL XML /Web Services (HTTP/SOAP) APIs in Java, C++, and C# JTA Transactions Management APIs – JMX
5 Key Features
27 3. Data Distribution Model Consistency Model Distribution without ACKs With ACKs With global locking Reliable Pub-Sub semantics Novel declarative "role" based model to ensure message delivery Dealing with Slow subscribers, net/node failures, etc Multiple transports TCP, Reliable UDP MultiCast
5 Key Features
28 4. Integration with External Data Sources A Framework for read-through, write-through, and write-behind Custom Connectors link to DBMS, Middleware Messaging, EII solutions Components - Loaders, Writers and Listeners
5 Key Features
29 Dealing with high event rate RFID streaming data, etc Many clients can subscribe to portions of the Data Fabric Express interest using complex query CQ engine continuously analyses data events Continuously calculates how the query view has been impacted Ships "delta" and merges into client result set. Notifications delivered with guaranteed latency threshold 5. Continuous Analytics
Comments