Memory-Centric Active Storage
Persistent Memory is redefining how we think about storage systems. By keeping durable data close to the CPU, efficient operations can be performed in-place without data movement. MCAS is an advanced new storage system that evolves the key-value paradigm to unleash this potential.
MCAS is an advanced in-memory object store designed from the ground up to leverage Persistent Memory and RDMA network technology. The result, unprecedented performance for operations on durable multi-TB data.
At the core of MCAS is a sequentially consistent sharded data architecture. Except for large DMA transfers, put, get and invoke operations are made synchronously. That is, when an operation returns, the data has been fully transferred and (in the case of write) made fully persistent.
- Real-time analytics and unpredictable workloads
- Distributed durable metadata (e.g., as a replacement for etcd)
- Data aggregation (e.g. Deep Learning parameters)
- Over 20M synchronous random read IOPS at less than a 7usec round-trip.
- Over 10M synchronous random write IOPS at less than 10usec round-trip.
- Immediate consistency on all writes
- Support for basic key-value API (C++, Python).
- Plugin architecture for Active Data Objects that allow deployment of data-specific operations executable in-store (C++, Python and other languages).
- Deployable on both plain TCP/IP sockets and high-performance RDMA (ibverbs).
- Zero-copy transfer of data when deployed on RDMA networks, supporting GPU-direct and other direct to device DMA technology.
- Fast primary index based on a crash-consistent hash table.
- Support for optional secondary indexes for range scans etc.
- Support for DRAM (without durability) and Intel Optane DC Persistent Memory.
"As soon as consistency models become slightly difficult to understand for application developers, we see that they are ignored even if performance could be improved. The bottom line is that if the semantics of a consistency model are not intuitively clear, application developers will have a hard time building correct applications" - Andrew Tanenbaum, Maarten Steen