Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

"Something went wrong"

During the server relocation last night a node of the file cluster gave out, and took it's assets with it.

We are currently running with old and new assets. Right there in the middle is a "donut hole", which some of you might have noticed already because a profile picture is missing every now and then.

We are very sorry about that, but that's life.

What's also life is, that we have offsite backups of everything, so fear not, all is good.

However, the backup replay might take a while: we can do a pretty good job with the new uplink in downloading, but it is a bit more complicated than just replicating a volume. Every asset that's being pulled from the backup needs to be checked if it is still active, downloaded and verified. My guess is, that it will take anywhere between one to five days to bring back everything.

In the meanwhile, if there is a file you really really need now, because it's homework or otherwise extremely(!) important(!!), you can drop us a mail at team@soup.io with the file URI. We will then restore the file for you  manually. Did I mention it needs to be life-or-death important? ;)

Again, sorry for the inconvenience, the next iteration of the file cluster will certainly not be susceptible to this kind of data loss.

edit: those of you who can't open their soup: we are working on that. When your soup opens again, it might not have the design you expect. Please be patient, yours will come back.

edit: the user avatars are now completely restored, which should lead to better performance right away.

edit: the last two days we were very invested in setting up the new file cluster, which is now mostly done, and is still importing files from the old server and the backup. Over time, speed will improve and most important of all, it is built very redundantly. Yay! No more inconvenient data loss in the future.

Also, since we now have more available space at our hosting facility, some other long planned improvements will be implemented over the next days and weeks, all targeted towards acceptable speed and a little bit more stability ;)

Btw, did anybody notice that this little trashcan in edit mode looks like a crown? ... I think I need to go to bed now.
Reposted fromupdates updates

Don't be the product, buy the product!