Haystack is a system designed by Facebook to store its small files (pictures). The design idea of the system is to merge small files into a large file for storage, reducing the number of files and IO times when accessing files.
Beaver, Doug, et al. “Finding a Needle in Haystack: Facebook’s Photo
Storage.” OSDI. Vol. 10. 2010.
F4 is a replacement for haystack.
Muralidhar, Subramanian, et al. “f4: Facebook’s warm BLOB storage
system.” 11th USENIX Symposium on Operating Systems Design and
Implementation (OSDI 14). 2014.
Windows Azure Storage
Windows azure storage uses a set of architecture, and provides queue, blob and table services. It has exquisite architecture and learning value. Was also published some related papers and introduced the whole architecture. In addition, was published a separate paper to introduce the practice of erasure coding.
Calder, Brad, et al. “Windows Azure Storage: a highly available cloud
storage service with strong consistency.” Proceedings of the
Twenty-Third ACM Symposium on Operating Systems Principles. ACM, 2011.
Huang, Cheng, et al. “Erasure coding in windows azure storage.”
Presented as part of the 2012 USENIX Annual Technical Conference
(USENIX ATC 12). 2012.
LinkedIn open source its object storage system ambry, and published a paper on sigmod2016, which introduces the architecture of the whole system in detail.
Noghabi, Shadi A., et al. “Ambry: LinkedIn’s Scalable Geo-Distributed
Ozone: The object store on HDFS
From the JIRA issue of ozone, you can see the development progress and documents
Introduction of hortonworks
Weil, Sage A., et al. “Ceph: A scalable, high-performance distributed
file system.” Proceedings of the 7th symposium on Operating systems
design and implementation. USENIX Association, 2006.