Storj Opens Its Decentralized Storage Service Project to Beta
Storj Labs has released the beta of its open source namesake decentralized cloud object storage software alongside opening up beta access to its own implementation of that software with its decentralized cloud storage service Tardigrade. In an interview with The New Stack, Storj Labs Executive Chairman and Interim CEO Ben Golub explained that Storj follows in the footsteps of other household name tech companies that allow its members to profit by “sharing” their resources — in this case, their spare storage space.
“AirBnB is the largest hotel company without owning a single hotel room, and Lyft and Uber the largest taxi company without owning a single taxi. In our case, we’ve got a cloud storage service, but we built it by enabling people to rent out spare capacity on their drives. It turns out that the vast majority of drives out there are only about 25% full and it doesn’t take any more capital or electricity or people to run a drive at 75%, so people are excited to give us their spare capacity,” said Golub.
The user experience is the same way as a standard cloud provider, except the company claims its service is faster cheaper, ad far more secure. In the background, the user’s data is split up into lots of little pieces and encrypted and distributed all over the network.
Originally the brainchild of founder Shawn Wilkinson, Storj has gone through two other implementations before arriving at version 3 (V3). Golub explained that V1 was “more of a proof of concept,” while V2 “got really, really big really quickly,” with 150PB and more than 100,000 nodes, but failed to meet the expectations for it to grow beyond that. This set the company off on the path to the V3 network.
“V2 was designed in a way that didn’t deliver enterprise grade durability or performance and would not have scaled to the exabyte or beyond range. So, we learned a lot and we did a massive rewrite,” said Golub. “One of the things that we’ve learned is that we want to make sure that supply doesn’t outstrip demand too much. One of the reasons that people join the network to rent out their drives is because they’re going to get paid, and if there isn’t enough demand to fill them up, then they, then they leave. Now we’re growing more moderately and making sure that our network is designed to deliver that hundred percent durability and you know, 99.999 availability and blazing fast speed and now we’ve achieved that.”
With this latest rewrite, Storj has been in alpha testing and a company statement claims that its performance is now “comparable to Amazon S3 and other centralized cloud storage providers” with Tardigrade delivering upload and download speeds that are “more than double S3” with no file loss and file availability at over 99.95%.
As for how Storj achieves all of this, there are stringent requirements and an auditing process for all of its storage providers, or Storage Node Operators, which are made up of a mix of individuals with high quality drives at home to universities and data centers with spare capacity. No matter who they are, some minimum requirements to be met include a dedicated processor for each storage node service, a minimum of 500GB, 2TB of bandwidth per month, 5Mbps upstream and 25Mbps downstream, and a max of five hours downtime monthly. All of this is confirmed by way of a month-long test process, including an audit to confirm requirements, but Golub stressed that the network was designed in a way to assure its operation even when a large number of nodes are not operational.
“We built the network assuming that some of the nodes are bad, either running incorrectly or run by bad people, and so the network was designed to be really resilient to that. For example, when you upload to our network, the first thing that happens is it gets encrypted using keys that only you have, so that neither the people who are running the nodes nor even Storj itself can ever look at or understand your data,” said Golub. “It then gets split up into about 80 pieces, of which any 30 can be used to put your file back together. And then each of those 80 pieces goes to a different drive on the network, and each drive is run by a different person on a different power supply, on a different network connection and in a different part of the world. It means we’re really resilient to anything happening, whether it’s a storm or an earthquake or anything like that.”
With the move to beta, Storj Labs will begin welcoming select developers onto the network, with more than 10,000 developers on the waitlist soon able to begin deploying applications using Tardigrade as their storage back end, alongside all of the Storj Labs Open Source Partner Program members. For developers looking to integrate Tardigrade into their applications, they will be able to use the CLI and libuplink developer library in Golang, C, and on Android, using the Minio S3 gateway.
Feature image by Tanja Schulte from Pixabay.
