Distributed Storage: Seaweed FS

Alex Mikhalev
1 min readMar 7, 2021

I came across Seaweed FS, which I think is perfectly architectured:

  • Acknowledgement of existence of other technologies, so master metadata can be stored in Redis(Cluster), Cassandra, Etcd
  • Any use case I could think of covered — FUSE mount, HDFS, S3 API gateway, WebDAV, Async Backup into the cloud
  • Tiered Storage.
  • Cross-Datacentre replication (I don’t need it at home, but the number of databases/storages supporting cross DC replication out of the box can be counted on one hand)

Quickstart with a single master on one RPI4 server and volume server on another took 10 minutes. Why did I spend so much time going through edge fs, Ceph and Gluster FS? Now I need to figure out how to migrate GlusterFS to it, currently, I have 3 RPI4 nodes each with 128 GB USB stick and 1 TB external drives holding:

  1. Docker volume — 3 replicas over 3 microSD cards
  2. largerpivolumes — JBOD configuration in GlusterFS over all disks (12 TB)
  3. Fastvolume — JBOD over usb sticks 337G
  4. slow volume — replicate over 2 disks with one arbiter

Now I need to plan for 30 minutes to have the same volumes setup and more importantly figure out a way to move data from Gluster into seaweed fs.

The challenge: I don’t have a single large enough disk to hold all data. Any suggestions?

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

Alex Mikhalev
Alex Mikhalev

Written by Alex Mikhalev

I am a systems thinker with a deep understanding of technology and a methodological approach to innovation

No responses yet

What are your thoughts?