Or for selling the user-created content entrusted to your company, to a company doing ML training. (The metal "briefcase" suggests doing it like an illicit trafficker exchange, maybe meeting in the middle of a bridge, or in an abandoned industrial site.)
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by AstroJetson on Sunday April 14, 2024 @04:25PM
(#40034158)
Never underestimate the bandwidth of a station wagon full of tapes - from a lifetime ago when we moved full systems backups from East Aurora to Chicago.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by remram on Sunday April 14, 2024 @04:33PM
(#40034229)
This is still a common thing. Amazon has AWS Snowball [1] (up to 210 TB), Backblaze has Fireball [2] (96 TB), GCP has Transfer Appliance [3] (300 TB) and I'm sure there are others.
They ship it to you, you fill it up (over S3 API for example) and send it back.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by remram on Sunday April 14, 2024 @06:46PM
(#40035179)
My gut reaction is that either people are moving data to another of their locations, in which case they have disks they can ship with regular transporters, or they are moving to the cloud, and they can use the cloud's transportation service. I might be wrong though!
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by crote on Sunday April 14, 2024 @04:50PM
(#40034360)
I'm not too surprised. There aren't that many parties with 100PB of data looking to switch from on-prem to the cloud. If you haven't migrated in the last 8 years already, you're probably never going to.
Keeping that truck around isn't exactly cheap either. I reckon it reached its technical or economic lifespan and simply isn't worth replacing.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by EdwardCoffin on Sunday April 14, 2024 @04:10PM
(#40034028)
Jim Gray talked about this idea in an ACM Queue interview in 2001 [1] calling it a TeraScale SneakerNet box:
Another option is to send whole computers. I’ve been sending NTFS disks (the Windows file system format), and not every Linux system can read NTFS. So lately I’m sending complete computers. We’re now into the 2-terabyte realm, so we can’t actually send a single disk; we need to send a bunch of disks. It’s convenient to send them packaged inside a metal box that just happens to have a processor in it. I know this sounds crazy—but you get an NFS or CIFS server and most people can just plug the thing into the wall and into the network and then copy the data.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by elorant on Sunday April 14, 2024 @04:24PM
(#40034144)
Why would it cost that much? A 30TB nvme costs about $3k. You need 12 of those, so that’s like 36k. Add another $4k for the rest of the hardware and the case. Plus a little overhead since it’s a branded item, it shouldn’t be more than $50k.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by SloopJon on Sunday April 14, 2024 @04:20PM
(#40034109)
This could be useful for bootstrapping a shared-nothing cluster or a disaster recovery node. A comparable amount of space in a storage array might occupy a 2-4U chassis, which would be somewhat less convenient to transport.
This thing isn't going to be cheap, so you'd probably want a more frequent transfer to justify it. Maybe sneaker net database journals or backups every week?
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by Sakos on Sunday April 14, 2024 @04:08PM
(#40034017)
It's not that alien. We recently migrated to a different storage provider and used something like this to move 200TB of data. Seems like a fancier version of the Google transfer appliance (https://cloud.google.com/transfer-appliance/docs/4.0/overvie...).
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by elorant on Sunday April 14, 2024 @04:30PM
(#40034199)
A data center has backup plans. This is more for data on the go. Think of a fashion show where the production shoots tens of TBs in high resolution and they can’t afford to setup a server room because it would be impractical. You get one of these and you’re done.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by haunter on Sunday April 14, 2024 @05:25PM
(#40034625)
Film production crew although it doesn't have a video digital out. But you might not even need it if you are only after storage, 8K RedCode raw is around ~7TB / hours
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by raihansaputra on Sunday April 14, 2024 @11:47PM
(#40036903)
Not datacenter related but i recall someone had a job of being a data mule between California and Australia to ship dailies to the VFX studio (in Australia) by hopping on the flight every day. I can imagine something like this (smaller) to be carry-on.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by nyx on Sunday April 14, 2024 @04:45PM
(#40034319)
It's 200GbE. Assuming you could actually fully saturate both of those links, it would only take a couple of hours to copy off the entire capacity. I think you'd more likely be capped by the write speed of the destination storage?
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by userbinator on Sunday April 14, 2024 @04:10PM
(#40034029)
Unfortunately it's probably not meant for long-term storage, and thus they may be using the highest density flash available, which means relatively short retention.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by extraduder_ire on Sunday April 14, 2024 @04:19PM
(#40034105)
It'd be a convenient add-on for the $2 licensing (IIRC) it costs for the port, so you can use common displays. I'd expect displayport to be more likely to be added though.
Sadly, I think part of the HDMI licence means you can't put analogue outputs on a device with a HDMI port after a certain year. (2014, I think. It was before then that I read that)
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by Rinzler89 on Sunday April 14, 2024 @04:26PM
(#40034161)
> so you can use common displays
VGA capable displays ARE common where this thing will be used (racks, movie sets/studios, etc). Plus, those display outputs are for emergencies only as this will always be used headless all the time. You'll get your GUI video output from the thin client connected to that thing over the network.
>I'd expect displayport to be more likely to be added though.
Not in severs. You don't want to waste PCI-E lanes for driving a display output that will never be used and that server chip doesn't even have an iGPU anyway.
Plus, the VGA output is most likely only displays what the motherboard controller is doing and can't display the GUI of the OS running on the CPU.
Also, the VGA connector is miles more robust and fault tolerant than digital outputs where if the handshake fails you get no picture.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by Ekaros on Sunday April 14, 2024 @04:22PM
(#40034132)
On other hand, just imagine how awesome mediabox this would be? Not that anyone will ever use it as one, but you could fit enough content for very long time.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by bunabhucan on Sunday April 14, 2024 @05:47PM
(#40034770)
It takes 2.5 hours to fill. What are you supposed to do during that time, watch a movie on your phone?
Maybe there's a pro version with roku / disney+ / netflix etc. integration.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by ssl-3 on Sunday April 14, 2024 @04:17PM
(#40034081)
It's just a computer, and a fairly compact one at that.
To you, it may seem like a very fancy computer that is made of pure unobtanium.
To a person running the X-ray machine, it's just another computer, and not too dissimilar from the thousands of other computers they've seen already this week.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by datadrivenangel on Sunday April 14, 2024 @04:25PM
(#40034154)
My photographer mates have great stories about impressing airport security with the density of their gear, and of flagrantly getting away with 3-5 times the weight allowances for their carryon bags.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by burnte on Sunday April 14, 2024 @03:57PM
(#40033926)
It's a solution, not a part. Plug this into the LAN, copy data. It manages everything about the file storage internally, etc. With this much storage (the sheer number of drives) you need a dedicated server to keep the data moving.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by codingmoh on Monday April 15, 2024 @12:05AM
(#40036972)
I saw your very recent comment on EMRs /EHRs and that you would like to build a new one.
we are actually in the process of building one, we have a doctor on board and we also have our first users.
Our goal is to create the most fantastic user experience for primary Care practitioners.
Our existing MVP uses AI to to speed up clinical documentation process using speech text to speech. Transcribing the conversation, summarizing it, generating a doctor's letter.
We try to identify what actions the doctor recommended to the patients while talking to them. For instance, regenerate a laboratory referral in real time.
We are writing our yc application atm.
I can't see your email in your profile, but you should be able to see mine. I would love to talk to you.
Please drop me a message if you are still interested in building a new EMR.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by LeoPanthera on Sunday April 14, 2024 @03:55PM
(#40033914)
I wonder if that's a typo though. 200G ethernet does exist, as part of the "Terabit Ethernet" standard, but it's extremely rare and you'd have trouble finding anything to connect it to that could negotiate that fast.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by zettabomb on Sunday April 14, 2024 @04:23PM
(#40034137)
It's not "extremely rare" in the intended market. It'll still take you over 2 hours to fill this, if you manage to saturate both links (which you can't realistically do).
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by crote on Sunday April 14, 2024 @04:44PM
(#40034318)
Isn't 200G starting to become reasonable common for high-end enterprise gear?
Dual-port 10G/25G is fairly standard on servers these days, and SANs with multiple 100G ports aren't exactly rare either. I wouldn't be too surprised to see 200G or 400G ports on newer spine switches these days.
Re:WD Ultrastar Transporter: Briefcase with 368TB of NVMe SS...
by c_o_n_v_e_x on Sunday April 14, 2024 @08:41PM
(#40035878)
If anyone is wondering, the business case for this kind of hardware is by avoiding paying public internet data egress and other fees when transferring data between cloud providers or your own on-prem DC. Some Data center providers have their own fiber networks to enable transfers at rates below what you'd pay to transfer via public.
Seagate offers a similar solution but as a service.. Lyve. Their hardware uses "docking stations" and portable unit with storage.
As others have mentioned, Amazon has a similar device/service but it can be used only for transferring data into Amazon afaik.
Or for selling the user-created content entrusted to your company, to a company doing ML training. (The metal "briefcase" suggests doing it like an illicit trafficker exchange, maybe meeting in the middle of a bridge, or in an abandoned industrial site.)
They ship it to you, you fill it up (over S3 API for example) and send it back.
[1]: https://aws.amazon.com/snowball/
[2]: https://www.backblaze.com/cloud-storage/features/fireball-da...
[3]: https://cloud.google.com/transfer-appliance/docs/4.0/overvie...
There's the "Snowcone", the "Snowball", and, for those really big transfers, the "Snowmobile".[2]
[1] https://aws.amazon.com/snow/
[2] https://www.youtube.com/watch?v=8vQmTZTq7nw&t=109s
Keeping that truck around isn't exactly cheap either. I reckon it reached its technical or economic lifespan and simply isn't worth replacing.
Another option is to send whole computers. I’ve been sending NTFS disks (the Windows file system format), and not every Linux system can read NTFS. So lately I’m sending complete computers. We’re now into the 2-terabyte realm, so we can’t actually send a single disk; we need to send a bunch of disks. It’s convenient to send them packaged inside a metal box that just happens to have a processor in it. I know this sounds crazy—but you get an NFS or CIFS server and most people can just plug the thing into the wall and into the network and then copy the data.
[1] https://queue.acm.org/detail.cfm?id=864078
It always amuses me reading old articles about things like data storage. These days you can get a 2TB SD card.
Who are you? What's your job? What kind of choices in life brought you to this point?
I really wonder, it's so alien.
This thing isn't going to be cheap, so you'd probably want a more frequent transfer to justify it. Maybe sneaker net database journals or backups every week?
Each Ethernet port is at 200 Gb, that is, 200 billion bits per second or about 20 billion bytes per second or 20 BBs.
Two such ports are 40 BBps
The 368 TB is 368,000 GB
So, the transmit time is at least
368,000 / ( 40 * 3600 ) = 2.556 hours
Sadly, I think part of the HDMI licence means you can't put analogue outputs on a device with a HDMI port after a certain year. (2014, I think. It was before then that I read that)
VGA capable displays ARE common where this thing will be used (racks, movie sets/studios, etc). Plus, those display outputs are for emergencies only as this will always be used headless all the time. You'll get your GUI video output from the thin client connected to that thing over the network.
>I'd expect displayport to be more likely to be added though.
Not in severs. You don't want to waste PCI-E lanes for driving a display output that will never be used and that server chip doesn't even have an iGPU anyway.
Plus, the VGA output is most likely only displays what the motherboard controller is doing and can't display the GUI of the OS running on the CPU.
Also, the VGA connector is miles more robust and fault tolerant than digital outputs where if the handshake fails you get no picture.
To you, it may seem like a very fancy computer that is made of pure unobtanium.
To a person running the X-ray machine, it's just another computer, and not too dissimilar from the thousands of other computers they've seen already this week.
we are actually in the process of building one, we have a doctor on board and we also have our first users.
Our goal is to create the most fantastic user experience for primary Care practitioners.
Our existing MVP uses AI to to speed up clinical documentation process using speech text to speech. Transcribing the conversation, summarizing it, generating a doctor's letter. We try to identify what actions the doctor recommended to the patients while talking to them. For instance, regenerate a laboratory referral in real time.
We are writing our yc application atm.
I can't see your email in your profile, but you should be able to see mine. I would love to talk to you.
Please drop me a message if you are still interested in building a new EMR.
https://www.fs.com/products/110530.html
200G switches are about the cost of a small car, but if your buying this thing that is probably not a showstopper:
https://www.wiredzone.com/shop/product/10023793-supermicro-s...
Dual-port 10G/25G is fairly standard on servers these days, and SANs with multiple 100G ports aren't exactly rare either. I wouldn't be too surprised to see 200G or 400G ports on newer spine switches these days.
Seagate offers a similar solution but as a service.. Lyve. Their hardware uses "docking stations" and portable unit with storage.
As others have mentioned, Amazon has a similar device/service but it can be used only for transferring data into Amazon afaik.