News for hackers. Stuff that matters.
Re:From Python to Lua: Why We Switched by kt9 on Tuesday January 26, 2016 @03:05PM (#10975616)
Hi, I'm the Founder at Distelli and I wrote the original version of the Distelli agent in Python that Brian rebuilt using Lua. The main reason I picked Python 2.4 is because we have large enterprise customers that are / were running Python 2.4 on servers and I had to support Python 2.4 to serve their use case.
Re:From Python to Lua: Why We Switched by batbomb on Tuesday January 26, 2016 @03:44PM (#10975910)
They were probably running RHEL5, but even in 2014 they would have been in the Production 3 phase which is really should only be used for legacy purposes, and that's definitely not the sort of system you push new software to. Red Hat even pushes those customers towards virtualization because a) hardware rarely lasts 7 years and b) they won't bother backporting drivers for new hardware to the old kernel at that point.

I still use a few RHEL5 machines but nobody would ever dream of pushing new software to them.

Re:From Python to Lua: Why We Switched by teh_klev on Tuesday January 26, 2016 @04:20PM (#10976169)
In the real world...if you have a large fleet of RHEL5 boxes/VM's, this a fact of life. We're slowly phasing out our CentOS 5 boxes, but you still need to be able to run new software on them.
Re:From Python to Lua: Why We Switched by hueving on Tuesday January 26, 2016 @06:19PM (#10977024)
>not the sort of system you push new software to.

Telling customers that they are doing things wrong isn't really a great way to win customers.

Re:From Python to Lua: Why We Switched by takeda on Tuesday January 26, 2016 @06:30PM (#10977090)
This is something I don't understand. If someone is using Java, they'll install the version of Java they need. If they use Ruby, they'll install the version of Ruby they need. If they need Lua, they'll do the same thing.

Why everyone is relying on python that's installed there for RedHat's needs (e.g. yum).

Unlike other languages, Python is probably the best one that can have multiple versions coexist together.

Re:From Python to Lua: Why We Switched by iofj on Tuesday January 26, 2016 @07:29PM (#10977377)
As a consultant I have very different experiences. Even in Java development you find shops that refuse to upgrade their JVM for various reasons.

RHEL 5 is really common.

Re:From Python to Lua: Why We Switched by ghshephard on Wednesday January 27, 2016 @09:22AM (#10980087)
A lot (all that I'm familiar with) of the enterprise software packages that I've worked with ship with their own, very specific (read: regression tested against) version of java. Enterprise software usually requires a ton of servers, so it's not like anything else on those servers will be using the jvm.
Re:From Python to Lua: Why We Switched by teh_klev on Tuesday January 26, 2016 @11:24PM (#10978193)
> Why everyone is relying on python that's installed there for RedHat's needs

Environments where the build of your server is strictly controlled and conservative...banks, mission critical apps, that kinda thing where stability from upstream is hugely important.

Re:From Python to Lua: Why We Switched by voltagex_ on Wednesday January 27, 2016 @01:11AM (#10978532)
AFAICT if you want support from Red Hat, you better not mess with the system configuration. It's probably best not to mess with too much on your production boxes anyway (maybe that's why Docker is so popular)
Re:From Python to Lua: Why We Switched by nine_k on Tuesday January 26, 2016 @05:55PM (#10976897)
How about running a (newer) local version of Python, without of course touching the system Python?

To my mind, it's usually a good idea not to use system Python, both packages and the interpreter, if you're not also controlling them. This makes building your deployables more involved, but makes the actual deployment more painless.

Re:From Python to Lua: Why We Switched by brownbat on Tuesday January 26, 2016 @06:49PM (#10977174)
> I had to support Python 2.4... large enterprise customers that are / were running Python 2.4 on servers...

How did the switch to Lua not break this in the same way?

Sorry for the naive question, I'm just not following how it can be possible to change languages, but impossible to change versions. At first glance, the latter seems like a special case of the former.

Re:From Python to Lua: Why We Switched by kt9 on Tuesday January 26, 2016 @07:24PM (#10977349)
Shipping it as a single download with zero dependencies meant that we would not have to worry in the future about what else was installed on the machine and the versions of system packaged the customers were running.
Re:From Python to Lua: Why We Switched by oblio on Wednesday January 27, 2016 @05:14AM (#10979262)
Couldn't you just bundle Python instead?
Re:From Python to Lua: Why We Switched by iofj on Tuesday January 26, 2016 @07:23PM (#10977345)
Lua and Luajit are small embeddable languages. They link into a single executable, including if you want the program they'll run.
Re:From Python to Lua: Why We Switched by orf on Tuesday January 26, 2016 @04:14PM (#10976125)
That's fair enough, but it sounds like you've constrained yourselves and your code for the sake of a few enterprise customers, and now you've had to clear your technical debt with a complete rewrite. Even if batbomb were right I would argue that any solution is better than supporting 2.4. The very minimum you would want is 2.5 at a stretch.
Re:From Python to Lua: Why We Switched by bb88 on Tuesday January 26, 2016 @03:42PM (#10975886)
In that case, Lua makes a lot of sense, frankly. I can write 100 lines of code in perl, but trying to get that deployed on mac, windows and linux can be a nightmare, especially if its using a library external to perl.

(EDIT: LUA->Lua)

Re:From Python to Lua: Why We Switched by glibgil on Tuesday January 26, 2016 @03:56PM (#10975997)
LUA? Still?
Re:From Python to Lua: Why We Switched by etiene on Wednesday January 27, 2016 @04:52PM (#10983779)
Lua used to be LUA, you'll find LUA written in old source code comments :) Don't be so mad about the Lua Uppercase Accident :)
[deleted]
Re:From Python to Lua: Why We Switched by talloaktrees on Tuesday January 26, 2016 @04:14PM (#10976124)
I think he referring to LUA vs Lua
Re:From Python to Lua: Why We Switched by christianbryant on Tuesday January 26, 2016 @11:56PM (#10978314)
I'm a Python devotee, but I appreciate the anecdote. I'm curious whether the switch 1) cost any features/algorithms and 2) whether the anticipated performance improvement really panned out since the switch, or if the gain wound up being simply architecture/deployment-related. Stepping back, did the whole re-write really balance out with all the work that had to be done, or could you have stuck with Python and simply solved the problems at hand?
Re:From Python to Lua: Why We Switched by brimworks on Wednesday January 27, 2016 @01:11AM (#10978533)
1) All features/algorithms were ported, and we only added new functionality that never existed in the old Python code base. A few of the features that got added include: dispatching builds, getting live logs, re-encrypt keys, deploy encrypted artifacts, fix many bugs, support running multiple agents on a single computer, etc. I should also mention that we previously had two separate downloads: a CLI used for uploading releases, and the agent used for dispatching deployments. The functionality of both of these code bases where merged into our new Lua based "client" so customers only have to install a single binary that is less than 10MB (the binary size depends on your platform).

2) Deployments with the new agent were noticeably faster, (like 1 second deployments that used to take 20+ seconds). However, this speed up was largely due to the Python codes design which had a "continuation" loop which was polling based, but the Lua code used coroutines and simply continued the threads when the steps were complete.

Overall, I think the choice to use Lua (or more specifically luvi) was a great decision. Note that it did take a few iterations in order to come up with the optimal way of using Lua + libuv. Originally I was taking the Node.JS approach of using callbacks, but that approach has two problems:

[a] It is difficult to properly implement "back pressure" so that all queues between producer and consumer are bounded.

[b] It is difficult to handle errors properly.

Lua coroutines allowed me to write a simple "green thread" library that encapsulated these challenges.

Re:From Python to Lua: Why We Switched by techie128 on Tuesday January 26, 2016 @05:52PM (#10976882)
Couldn't you have run two versions of Python at the same time on their systems? Would it have been possible to ship a Python version with your software?
Re:From Python to Lua: Why We Switched by collyw on Thursday January 28, 2016 @11:31AM (#10989134)
easily
Re:From Python to Lua: Why We Switched by Pharaoh2 on Tuesday January 26, 2016 @04:09PM (#10976095)
"I had never seriously used Python before joining Distelli, so there was a bit of a learning curve"

That was your problem, you just made a huge mistake and depending on how big your python code base was, you have now alienated most of your developers by moving to a language they are not already comfortable with.

Do your customers already have lua installed on their servers or are you bundling it now? You could have just a easily bundled a newer python interpreter. Lua is a nice scripting language, if you want to embed it into another app or to create a plugin system. It's not a nice general purpose scripting language, mostly because of the vast pypi repo.

Re:From Python to Lua: Why We Switched by bigtunacan on Tuesday January 26, 2016 @05:05PM (#10976525)
How is this alienating anyone? This is a command line tool. The end users are suddenly using a different language as a result. Their install process was streamlined and system dependencies were removed by including it in the install package.

The same thing could have been achieved within Python, but from an end user perspective I'm guessing this is an improvement for most of their customers.

Re:From Python to Lua: Why We Switched by Pharaoh2 on Tuesday January 26, 2016 @05:30PM (#10976728)
It's alienating Distelli developers. The reason cited for the move is totally unacceptable in terms of the new skillset that their developers will need to acquire to continue development of the tool.

The cost of effort their developers will have to put in to be as effective as a LUA developer as they were as a python developer is really not worth this move. There are more to moving to a different platform/language than just the end users.

Re:From Python to Lua: Why We Switched by rifung on Tuesday January 26, 2016 @06:32PM (#10977101)
They only have a few developers, so it's really not a big deal. I think they only have 2 or 3, including the CEO who, obviously, has a lot of other things to do than write code.

Being as the size of the team is so dependent on a small number of high performers, I don't think it's unusual that they picked a language one of the main people in the company is extremely fluent in.

I've worked with Brian before and he is a beast. I don't know if 10x engineers exist, but if they do I would certainly consider him one.

Re:From Python to Lua: Why We Switched by devishard on Wednesday January 27, 2016 @02:57AM (#10978840)
Describing him as a 10x developer makes me even more skeptical of the decision to switch. Developers who deliver 10x as many features inherently do so at the cost of maintainability.
Re:From Python to Lua: Why We Switched by rifung on Wednesday January 27, 2016 @03:36AM (#10978952)
> Developers who deliver 10x as many features inherently do so at the cost of maintainability

I don't think what you say is completely true. People who are more experienced tend to be able to produce features much more quickly than those who are junior, just because they've already run into issues before so can anticipate them or solve them more quickly. If anything I would argue people like that actually write code more quickly that is more maintainable.

I don't know if he can ship 10x as many features; I didn't think term 10x engineer was used literally. I just know I've worked with quite a number of people and he is one of the best developers I know.

Re:From Python to Lua: Why We Switched by Chris2048 on Wednesday January 27, 2016 @07:56AM (#10979745)
> he is one of the best developers I know

But then, we'd need to know how good your assessment skill/method is. Maybe you misjudge all the developers you work with? What feedback do you get?

Re:From Python to Lua: Why We Switched by brimworks on Wednesday January 27, 2016 @10:16AM (#10980404)
@rifung - Thanks for the kind words.

@Chris2048 - Distelli is hiring. If you want to make your own assessment of my skill, then join us :).

Re:From Python to Lua: Why We Switched by Sanddancer on Tuesday January 26, 2016 @06:08PM (#10976974)
The end users are the ones that keep the lights on though. It sounds a lot like that Python was more and more of an anchor around the team, and the costs of migration were smaller than the costs of the other options. Yes, some developers were probably upset that they couldn't use their preferred language anymore. However, it sounds like because they're now using a language designed to be embedded, they can focus a lot more on solving problems than fighting with a language that has a lot more platform-specific behavior.
Re:From Python to Lua: Why We Switched by fsloth on Wednesday January 27, 2016 @02:09AM (#10978706)
"It's alienating Distelli developers."

Are you in a position to speak for Distelli developers?

"The cost of effort their developers will have to put in to be as effective as a LUA developer as they were as a python developer is really not worth this move."

I suppose this could be true in a poisonous environment where scheduling does not accommodate the need to learn a new skillset.

Unless you are familiar with the employer this is an incredibly hostile presumption. I.E. I would not work for such an employer nor would I want to be their customer

Re:From Python to Lua: Why We Switched by Chris2048 on Wednesday January 27, 2016 @08:05AM (#10979783)
What does accommodating for a new skillset make a difference to the cost?

There will still be a minimum effort (e.g. in hours) that cannot be reduced by the nature of the employer. A dev will still have to put in the hours to learn a new language.

It seems the cost being described here is cost to the employer, not to the developer.

Re:From Python to Lua: Why We Switched by etiene on Wednesday January 27, 2016 @04:57PM (#10983820)
I use Lua as a general use language and I'm incredibly satisfied. It's an amazing language.

I also need to mention that Lua is the easiest language I've ever encountered, so any competent developer can learn it quickly, it's not alienating anyone.

Given that, I agree that LuaRocks isn't so vast as other languages' repositories and this can be a drawback of using Lua. But why again are you judging a language by it's package manager? If it's a good language wouldn't it be better to suggest more people to write packages instead of suggesting people not to use it?