Re:Using Python helped YouTube ship features faster than Goo...
by nitwit005 on Monday March 26, 2018 @12:05AM
(#16676516)
As much as I want to credit python, I doubt that was the cause. It's typically far easier to ship a new feature at a small company, because there's no bureaucracy, no wait for QA, no need to translate text into 20 languages, etc.
Re:Using Python helped YouTube ship features faster than Goo...
by justonepost on Monday March 26, 2018 @04:37AM
(#16677508)
Startups are generally more tolerant of failure and half baked code. Googlers, who have lived there whole lives achieving perfection in school, not so much. I see this all the time at big companies that hire 4.0 GPA from ivy leagues.
Re:Using Python helped YouTube ship features faster than Goo...
by tomgp on Monday March 26, 2018 @04:39AM
(#16677514)
I think there's certainly something to be said for the agility of small teams in shipping new features but I also think it's not particularly contentious to suggest that developers are more productive in higher level languages esp. when your focus is on user experience etc.
Re:Using Python helped YouTube ship features faster than Goo...
by hyperpallium on Monday March 26, 2018 @12:59AM
(#16676719)
Yes. Perhaps also brilliant engineers don't want to hack together something half-assed that barely works, but does provide the feature.
I've always liked that youtube uses PHP - much derided, but its declarative "templating" is the easiest, simplest, most direct way to generate html. Maybe that was a factor too.
But I think the youtube engineers might have a more accurate opinion on why they were quicker, than the google video engineers.
Re:Using Python helped YouTube ship features faster than Goo...
by grzm on Monday March 26, 2018 @01:06AM
(#16676751)
> but its declarative "templating"
Do you have any insight onto how much of the templating aspect of PHP is used at Youtube? I know of some large PHP installations where PHP is used as a general programming language that does produce web pages, but the templating aspect of PHP is pretty much minimal, and other engines actually produce the final output.
Re:Using Python helped YouTube ship features faster than Goo...
by DonHopkins on Monday March 26, 2018 @04:40AM
(#16677520)
For a so-called templating language, PHP sure falls short, and thus has a plethora of ad-hoc, informally-specified, bug-ridden, slow implementations of half of many other templating languages implemented in it.
Re:Using Python helped YouTube ship features faster than Goo...
by DonHopkins on Tuesday March 27, 2018 @04:20AM
(#16686399)
Yet Python has never claimed to be a templating language itself, let alone "the easiest, simplest, most direct way to generate html".
My point is that PHP pretends to be a templating language, then falls flat on its face, so people end up implementing much worse (Smarty, MediaWiki) or sometimes even better (Twig) templating languages on top of it.
In other words, it's simply not true that PHP's "declarative "templating" is the easiest, simplest, most direct way to generate html", or there wouldn't be a need for Smarty, the MediaWiki abomination (which is 17 years old but still widely used on 29 million pages), or Twig.
I'd say "to hack together something half-assed that barely works, but does provide the feature" perfectly describes the PHP development process itself.
Re:Using Python helped YouTube ship features faster than Goo...
by m45t3r on Monday March 26, 2018 @07:51AM
(#16678137)
Remember, YouTube born 13 years ago. That time, online code editing and hot reloading were big things only archived with dynamic languages like Python and Ruby.
Re:Using Python helped YouTube ship features faster than Goo...
by dbspin on Monday March 26, 2018 @03:11AM
(#16677197)
As someone who was creating a reasonably popular 'video podcast' back then, I have a different perspective...
Uploading to Google video was incredibly slow relative to Youtube. During the period of the 'battle' between the two services, Youtube allowed only very short videos - excluding most useful content, but discouraging piracy (except for very laborious multi part uploads). G'Video on the other hand allowed uploading of essentially any length, although at much lower resolution
But it was incredibly laborious, as uploads took hours - and were liable to fail without warning (at least uploading from Europe). I recall there was also an emphasis on adding lots of metadata etc, another barrier to uploading.
The social elements of youtube - perhaps in hindsight addicting rather than useful, are rightfully acknowledged. Whats less well remembered is the sheer clunkiness of using Google Video. The service looked and responded much more like a corporate intranet than a website. Still, I - and lots of other 'vidcasters' at the time were sad to see it go.
"While sites like Youtube and Guba, may or may not have a future primarily as redistributors of broadcast content, they’ve done little to foster the creation of original work. In fact, by restricting the length and size of files which can be uploaded (ostensibly to reduce copyright infringement), YouTube have diminished their chances of becoming a hotbed of original content. Google video, although bravely eschewing any restrictions on the length of uploaded content (whilst foolishly restricting video quality to an extremely low bit rate), does little to foster the community creation or pooling of talent needed to inspire the development of original shows and films. Note, it’s far from clear that it was ever Google’s intention to become a generator of new IP, so Google Video shouldn’t necessarily be seen as a failure"
Re:Using Python helped YouTube ship features faster than Goo...
by justonepost on Monday March 26, 2018 @04:41AM
(#16677525)
Yeah, absolutely. I remember that being a big thing as to what Google Video said when they bought youtube. The immediacy of the upload was a huge factor in YouTube's success. It's not hard to imagine that YouTube won via better PM. That often happens.
Re:Using Python helped YouTube ship features faster than Goo...
by bsder on Monday March 26, 2018 @03:05AM
(#16677171)
YouTube was also burning money for bandwidth like nobody's business. I suspect that Google Video tracked and limited bandwidth.
If Google hadn't bought them, YouTube were headed straight for bankruptcy. People forget that the YouTube deal was a big time boardroom back scratch between Google and the VC's.
The worst part is that Google subsidizing YouTube has set back a real monetization of video on the web.
Re:Using Python helped YouTube ship features faster than Goo...
by nandwidth on Monday March 26, 2018 @04:40AM
(#16677516)
I disagree with that. I think the web is monetizing video at an appropriate rate, considering most video itself is essentially worthless.
That video content is costly to produce, and kills bandwidth indiscriminately, does not mean video is valuable. Just because there’s a high bar for video to exist, doesn’t mean that cost should get passed on to consumers.
Most video is non-essential, and consumers get to be picky about where their disposable income goes. Video needs to be redeeming to the viewer, whatever their tastes may be, and whose fault is it that the surplus of junk content exists?
Re:Using Python helped YouTube ship features faster than Goo...
by otakucode on Monday March 26, 2018 @04:53PM
(#16682490)
I'm not sure how YouTubes current existence given the past decade or so isn't a huge antitrust violation. They've run the platform at a massive loss, subsidized by their other business ventures to make it impossible for anyone to create a competitor. I can't imagine that if you propose to a VC that you need to spend hundreds of millions of dollars on building infrastructure that you will then need to run at a loss potentially for decades before hoping to see any profit it would go well.
Re:Using Python helped YouTube ship features faster than Goo...
by jakeogh on Monday March 26, 2018 @04:22AM
(#16677469)
I remember when Google announced video.google.com was closing, and then they backtracked and said it wasn't. And then they closed it. I have video.google.com posts that I would like to migrate, but it's gone.
Re:Using Python helped YouTube ship features faster than Goo...
by tjoff on Monday March 26, 2018 @05:13AM
(#16677625)
Yeah, no. That is obviously not why, and it is quite telling that the guy being interviewed still, after the purchase, hasn't got the slightest clue as to why (or doesn't want to admit it).
It is disheartening to see this on the front page with so little substance.
Re:Using Python helped YouTube ship features faster than Goo...
by peoplewindow on Monday March 26, 2018 @02:50AM
(#16677110)
I worked at Google during that period. It's not quite as simple as "Python > C++" although yes, for sure, you're going to produce new UI features at a far faster rate working in a high level language like Python than working with C++. Absolutely that was a factor. I don't think Google Video had hundreds of developers vs 20 though. That doesn't sound right to me.
The real reasons YouTube won were strategic, not purely a function of developer speed.
In particular YouTube focused a lot on social / discovery features and prioritised them, at a time when the importance of social wasn't so obvious and Google in particular was very bad at anything with a social dimension. So Google Video had super scalable backends, great search etc, but it wasn't very good at user profiles, commenting, channels, subscribers, discovery, content surfacing and so on. Whereas YouTube excelled at these things and was good at encouraging people to upload whatever random vids they created even if apparently worthless.
Google in contrast was less certain that free user generated content would ever be a big deal, and right from day one saw Google Video primarily as a marketplace. It focused much more on acquiring content rights from professional producers as a result. Even YouTube wasn't really sure UGC+ads was going to turn into a real business which is why they had a lax approach to video piracy for so long. Early YouTube traffic was to some extent driven by piracy of professional content, a problem that only got wiped out once Google acquired them and build content id. Even when comments were eventually added, it was a tiny box to the side of the video which took up the bulk of the web page, and there was an admonition "Please make sure your comments are useful and informative" - very Google. YouTube on the other hand made the video less prominent and what users were saying about it much more so.
Social features are a great use case for Python because you need to churn them out and iterate very fast, the cost of correctness bugs is very low, and their performance complexity is not very high, so that's the sweet spot for a scripting language. On the other hand around the time YouTube were acquired they struggling tremendously with the scaling aspects of their operation and large chunks of the site had to be quickly switched to C++. YouTube when I left was still a mix of Python and C++, with the C++ code handling things like search, transcoding, thumbnail serving, anti-spam, content id, video serving and so on. All the heavy lifting. Python was left handling the main site UI, admin tooling and not much else.
There were also some technical mis-steps by the Google Video team. YouTube relied entirely on Flash from day one. For whatever reason the GV team were far more reluctant to hitch themselves to the Flash plugin and for example they only enabled video upload via a web form about a year after YouTube entered beta, before that you had to use a desktop app to do it. They also wrote their own video playing plugin but it wasn't as good as Flash and they eventually scrapped it. Their paid for pro content also required a dedicated video player app. YouTube being a small startup couldn't get content deals as it was too tiny for the producers to deal with, so they just focused 100% on UGC and features like embedding, which acted as a giant advert for their service.
Re:Using Python helped YouTube ship features faster than Goo...
by bb88 on Monday March 26, 2018 @09:24PM
(#16684348)
You say:
> Social features are a great use case for Python
And then say:
> On the other hand around the time YouTube were acquired they struggling tremendously with the scaling aspects of their operation and large chunks of the site had to be quickly switched to C++.
I fail to see how python failed youtube here.
You use the right tools for the job. And the tools that got you 10e1 views per day may not be the tools that get you to 10e10 views per day.
I once worked at a place that supposedly "engineered" their backend to handle 1,000,000 connections at once. Only everything fell over at 10,000 connections in production, and they struggled for two years to get to 100,000 connections.
The point is, you don't know what you don't know, and you won't know the pain points are until you hit them. And the more groundbreaking your service (in this case internet video distribution), the chance of your architecture being 100% correct from the start is zero.
Re:Using Python helped YouTube ship features faster than Goo...
by candiodari on Monday March 26, 2018 @03:47AM
(#16677322)
If I recall correctly youtube also focused on embedding early on. Embedding google video videos was impossible. This lead a lot of people to youtube and google videos just missed out on it.
Re:Using Python helped YouTube ship features faster than Goo...
by swyx on Monday March 26, 2018 @01:13PM
(#16680535)
thank you for sharing your insider view. I'm just a casual observer, so I do feel a little guilty posting this story here but it is a firsthand source.
Re:Using Python helped YouTube ship features faster than Goo...
by anabis on Monday March 26, 2018 @01:36AM
(#16676850)
I heard an anecdote where the actual first movers on a new video feature was porn sites. They were smaller still, and probably had a more tolerant audience.
Re:Using Python helped YouTube ship features faster than Goo...
by swyx on Monday March 26, 2018 @09:01AM
(#16678499)
I feel like entertainment (gaming, porn, even say tourism for spaceflight) has always punched above its weight in leading technology. Really makes you think about whether we are more motivated to use our intelligence/resources to solve hard problems or to please ourselves.
Additionally, people are more tolerant of mistakes made by small companies. Features stopped working? Maybe couple of people will complain.
Google's features stopped working? Media, bloggers.. everyone will have an opinion on why this happened and how to resolve it.
Anyway, I agree with your conclusion: big companies move really slowly in comparison with smaller ones.
I've always liked that youtube uses PHP - much derided, but its declarative "templating" is the easiest, simplest, most direct way to generate html. Maybe that was a factor too.
But I think the youtube engineers might have a more accurate opinion on why they were quicker, than the google video engineers.
Perhaps you are thinking of Facebook?
Do you have any insight onto how much of the templating aspect of PHP is used at Youtube? I know of some large PHP installations where PHP is used as a general programming language that does produce web pages, but the templating aspect of PHP is pretty much minimal, and other engines actually produce the final output.
https://www.smarty.net/
https://en.wikipedia.org/wiki/Wikipedia:Templates
https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule
My point is that PHP pretends to be a templating language, then falls flat on its face, so people end up implementing much worse (Smarty, MediaWiki) or sometimes even better (Twig) templating languages on top of it.
In other words, it's simply not true that PHP's "declarative "templating" is the easiest, simplest, most direct way to generate html", or there wouldn't be a need for Smarty, the MediaWiki abomination (which is 17 years old but still widely used on 29 million pages), or Twig.
I'd say "to hack together something half-assed that barely works, but does provide the feature" perfectly describes the PHP development process itself.
Uploading to Google video was incredibly slow relative to Youtube. During the period of the 'battle' between the two services, Youtube allowed only very short videos - excluding most useful content, but discouraging piracy (except for very laborious multi part uploads). G'Video on the other hand allowed uploading of essentially any length, although at much lower resolution
But it was incredibly laborious, as uploads took hours - and were liable to fail without warning (at least uploading from Europe). I recall there was also an emphasis on adding lots of metadata etc, another barrier to uploading.
The social elements of youtube - perhaps in hindsight addicting rather than useful, are rightfully acknowledged. Whats less well remembered is the sheer clunkiness of using Google Video. The service looked and responded much more like a corporate intranet than a website. Still, I - and lots of other 'vidcasters' at the time were sad to see it go.
Here's a perspective from the time - from a blogpost I wrote about different video services in 2006 (https://garethstack.com/2006/07/24/the-unavoidable-future-of...)...
"While sites like Youtube and Guba, may or may not have a future primarily as redistributors of broadcast content, they’ve done little to foster the creation of original work. In fact, by restricting the length and size of files which can be uploaded (ostensibly to reduce copyright infringement), YouTube have diminished their chances of becoming a hotbed of original content. Google video, although bravely eschewing any restrictions on the length of uploaded content (whilst foolishly restricting video quality to an extremely low bit rate), does little to foster the community creation or pooling of talent needed to inspire the development of original shows and films. Note, it’s far from clear that it was ever Google’s intention to become a generator of new IP, so Google Video shouldn’t necessarily be seen as a failure"
If Google hadn't bought them, YouTube were headed straight for bankruptcy. People forget that the YouTube deal was a big time boardroom back scratch between Google and the VC's.
The worst part is that Google subsidizing YouTube has set back a real monetization of video on the web.
That video content is costly to produce, and kills bandwidth indiscriminately, does not mean video is valuable. Just because there’s a high bar for video to exist, doesn’t mean that cost should get passed on to consumers.
Most video is non-essential, and consumers get to be picky about where their disposable income goes. Video needs to be redeeming to the viewer, whatever their tastes may be, and whose fault is it that the surplus of junk content exists?
Um, I think you are confusing cause and effect.
Isn't worthless video exactly the problem that subsidizing video delivery caused?
If you have to pay even a penny to put your video on the web, most video disappears and suddenly the quality skyrockets.
unrelated: the ui is impossible to navigate on mobile.
https://www.youtube.com/watch?v=H542nLTTbu0
also I first heard this story on the Talk Python podcast https://talkpython.fm/episodes/show/156/python-history-and-p...
Of course its not the full story but it is an interesting comment and from a firsthand source. That's all I know, I dont have a dog in this fight :)
It is disheartening to see this on the front page with so little substance.
Commenters I’m this thread: “nah it wasn’t python.”
The real reasons YouTube won were strategic, not purely a function of developer speed.
In particular YouTube focused a lot on social / discovery features and prioritised them, at a time when the importance of social wasn't so obvious and Google in particular was very bad at anything with a social dimension. So Google Video had super scalable backends, great search etc, but it wasn't very good at user profiles, commenting, channels, subscribers, discovery, content surfacing and so on. Whereas YouTube excelled at these things and was good at encouraging people to upload whatever random vids they created even if apparently worthless.
Google in contrast was less certain that free user generated content would ever be a big deal, and right from day one saw Google Video primarily as a marketplace. It focused much more on acquiring content rights from professional producers as a result. Even YouTube wasn't really sure UGC+ads was going to turn into a real business which is why they had a lax approach to video piracy for so long. Early YouTube traffic was to some extent driven by piracy of professional content, a problem that only got wiped out once Google acquired them and build content id. Even when comments were eventually added, it was a tiny box to the side of the video which took up the bulk of the web page, and there was an admonition "Please make sure your comments are useful and informative" - very Google. YouTube on the other hand made the video less prominent and what users were saying about it much more so.
Social features are a great use case for Python because you need to churn them out and iterate very fast, the cost of correctness bugs is very low, and their performance complexity is not very high, so that's the sweet spot for a scripting language. On the other hand around the time YouTube were acquired they struggling tremendously with the scaling aspects of their operation and large chunks of the site had to be quickly switched to C++. YouTube when I left was still a mix of Python and C++, with the C++ code handling things like search, transcoding, thumbnail serving, anti-spam, content id, video serving and so on. All the heavy lifting. Python was left handling the main site UI, admin tooling and not much else.
There were also some technical mis-steps by the Google Video team. YouTube relied entirely on Flash from day one. For whatever reason the GV team were far more reluctant to hitch themselves to the Flash plugin and for example they only enabled video upload via a web form about a year after YouTube entered beta, before that you had to use a desktop app to do it. They also wrote their own video playing plugin but it wasn't as good as Flash and they eventually scrapped it. Their paid for pro content also required a dedicated video player app. YouTube being a small startup couldn't get content deals as it was too tiny for the producers to deal with, so they just focused 100% on UGC and features like embedding, which acted as a giant advert for their service.
More insights here:
https://googlesystem.blogspot.ch/2006/08/why-is-youtube-more...
> Social features are a great use case for Python
And then say:
> On the other hand around the time YouTube were acquired they struggling tremendously with the scaling aspects of their operation and large chunks of the site had to be quickly switched to C++.
I fail to see how python failed youtube here.
You use the right tools for the job. And the tools that got you 10e1 views per day may not be the tools that get you to 10e10 views per day.
I once worked at a place that supposedly "engineered" their backend to handle 1,000,000 connections at once. Only everything fell over at 10,000 connections in production, and they struggled for two years to get to 100,000 connections.
The point is, you don't know what you don't know, and you won't know the pain points are until you hit them. And the more groundbreaking your service (in this case internet video distribution), the chance of your architecture being 100% correct from the start is zero.
Python has some of the capabilities, but surely those libraries are also written in C/C++