News for hackers. Stuff that matters.
Re:The art of over-engineering your side projects by caleblloyd on Friday September 01, 2017 @08:31AM (#15147795)
Lots of times I use side projects as a way to explore new stacks, deployment approaches, etc. The goal of the project is not to deliver anything, it's to learn so that when I go to do my next "real" project at work, I know what works well and what doesn't.
Re:The art of over-engineering your side projects by alexandercrohde on Friday September 01, 2017 @10:10AM (#15148425)
Exactly. When I see a coworker start irrationally focusing on a new technology at work the first thing I think is "This person probably has no side projects to play with"
Re:The art of over-engineering your side projects by markcerqueira on Friday September 01, 2017 @12:26PM (#15149911)
> "This person probably has no side projects to play with"

Exploring new technologies that could be beneficial (or harmful) to the company shouldn't be a side project. Perhaps we can shift the thinking from "this person doesn't have time for side projects" to "how can we make exploring new technologies a part of the job?"

Re:The art of over-engineering your side projects by WalterSear on Friday September 01, 2017 @01:31PM (#15150589)
How about making wednesday mornings, a 4hr 'personal project' time? Make it a requirement that everyone has to work with tools at the edge of their skillset, that is also relevant to work, but let them choose what, why, how. They could work on a personal project, or experiment with a new library or tool they would like to incorporate.

Start the day with a "not really a standup", where people can quickly explain what they want to accomplish in the next four hours.

Once a month, you spend a couple of hours of the time demoing the work you have done for each other, explaining the technology and what you have learned, and then discussing and evaluating our individual technical weaknesses and deciding what to do next. This rest of this monthly meeting could also be spent improving social coding skills - code and commit hygiene, team processes etc.

Not only would this keep people technically sharp but it keeps them from playing with toys in production and breaks up the work week, while still fulfilling company goals.

Re:The art of over-engineering your side projects by jeffdavis on Saturday September 02, 2017 @03:03PM (#15157643)
If you have to start giving reports, it kills creativity. The point is to do something everyone else may think is irrational, but you feel would lead somewhere interesting.
Re:The art of over-engineering your side projects by WalterSear on Friday September 01, 2017 @10:00PM (#15153776)
This also lets junior developers practice planning and self-management, since they basically have 12 hours to come up with something to show.
Re:The art of over-engineering your side projects by zeta0134 on Friday September 01, 2017 @02:36PM (#15151264)
We call this development time at my workplace. Every two weeks, I spend the first half of my shift doing whatever I please, with encouragement for it to be work related, but no real guidance.

Last week I wrote a utility in Python to parse a configuration file that we had so far been editing by hand, or with bad, slow and cumbersome tools. Now our Linux guys can make the changes is seconds across multiple machines, and the parser is much safer than the bash hacks we had been tempted to use before.

Point being, development time can be great! More companies should embrace the idea of letting their employees improve their own workflow. Some of our best ideas were just that: a little spark that an employee had the opportunity to actually explore.

Re:The art of over-engineering your side projects by Raphmedia on Friday September 01, 2017 @12:45PM (#15150136)
Depends. Using a new stack for the company's blog is fine. Forcing a new technology into the middle of an existing project isn't.
Re:The art of over-engineering your side projects by CodeCube on Friday September 01, 2017 @01:08PM (#15150347)
It doesn't even have to be something "productive" ... what's being proposed here is that companies invest in literally having developers research new technologies, without expectations that said research will result in anything immediately profitable.
Re:The art of over-engineering your side projects by pmalynin on Friday September 01, 2017 @01:41PM (#15150681)
Some companies are already doing this. When I worked at Intuit we would have Whitespace days and Innovation days, roughly every month or so where you could spend 1-3 days doing whatever you want.
Re:The art of over-engineering your side projects by shados on Friday September 01, 2017 @07:26PM (#15153095)
While I have a lot of freedom at work to explore side projects and random stuff and I spend a lot of time doing so, there is a part of me that feels like there's a reason software engineers, at least in the US, are paid so damn much. To me, it's kind of implied that I'm paid <ridiculous amount compared to virtually any other job with similar requiring similar amount of training> is because I'm going to be doing more than the 35-40whatever hours I spend at the office.

Part of that is support (though about half of the jobs I've had involved little to no on call duty), part of it is training one self, in addition to everything I have to do during the day.

Let's call it discretionary overtime baked in pay since we're usually salaried.

Re:The art of over-engineering your side projects by closed on Friday September 01, 2017 @12:37PM (#15150053)
Agreed. It seems like side projects for work are basically research (and research is important!).
Re:The art of over-engineering your side projects by shardinator on Friday September 01, 2017 @02:18PM (#15151074)
Or just say 20% of your time you can do what you want. Seems like a solved problem to me.
Re:The art of over-engineering your side projects by markcerqueira on Friday September 01, 2017 @04:46PM (#15152260)
Having a solution to the problem and having people advocate for implementing that solution in their companies are too different things.

We know about lots of best practices but if no one is advocating for them at their companies those practices we'll just be shining cities on a hill that we look to from a deep, dark valley.

Re:The art of over-engineering your side projects by Walkman on Friday September 01, 2017 @11:01PM (#15153981)
We actually do this. We plan the learning phase for the sprint and we time box these kind of things for 1-2 days.
Re:The art of over-engineering your side projects by shados on Friday September 01, 2017 @11:00AM (#15148873)
100% this.

My side projects are absurdly over engineered and I get nothing done. But I learn what not to do when it matters.

Re:The art of over-engineering your side projects by karrotwaltz on Friday September 01, 2017 @11:37AM (#15149359)
I recently built a C++ meta-programming monster for a side project. I want to make an article about that in the near future, and I am somewhat afraid that some people might think of reusing it in a real project.

I had a lot of fun doing it, and indeed learnt to not do it in prod, the compiler errors are a nightmare to read and good luck maintaining that in the long run.

Re:The art of over-engineering your side projects by mindfulplay on Friday September 01, 2017 @11:49AM (#15149510)
That seems like a great side project. The one that let's you understand things NOT to do. At the same time, it helps engender basic traits like curiosity, try-until-you-fail-and-try-again, engineering skills etc.

There are two types of side-projects though: one such as yours, where you try and understand a concept or engineer something just for the fun of it.

And there is second one: a game that will mature into something; a hobby project that will actually develop into a company. I think it's good to disambiguate these two as the OP was more about the latter where you may never get to a point to see it run or build let alone flourish into something meaningful - as overengineering kills the fail fast fail often mentality and the project itself.

Re:The art of over-engineering your side projects by jpfed on Friday September 01, 2017 @12:30PM (#15149971)
Maybe the real lesson here is that the last open task on your project is something that can present human-readable digests from parsing the compiler errors...
Re:The art of over-engineering your side projects by wellthen on Friday September 01, 2017 @12:02PM (#15149655)
My constant on-again/off-again side project is something similar but in C#. Spending a week building a preprocessor and then discarding it because I didn't need it was both infuriating and strangely pleasing at once. :)
Re:The art of over-engineering your side projects by taeric on Friday September 01, 2017 @11:11AM (#15149036)
To be fair, many of us have plenty of side projects that are just not computer related. Reality is those should also provide teaching opportunities. (Indeed, for many, teaching children is essentially a full time job in terms of time commitment.)

Still, I completely agree with the point that "experiments" should be kept off the critical path as much as possible. It all depends where your risk/reward is, though. The people that took the risk on a stack and got rewarded with success are highly visible. Just as many took the risk and failed. (Probably more?)

Re:The art of over-engineering your side projects by FooHentai on Saturday September 02, 2017 @04:16AM (#15154910)
Now that's a good point - I've noticed a tendency to use side projects as a buffer for frustrations being experienced at work. Maybe there is a fundamental amount of true engineering I need to do on a weekly basis; if I'm not getting my fix at work it'll spill over to hobbies.

Lately I'm pulling back from technical deep diving at work. A lot. My role as an enterprise architect (bleh, don't judge too much plz) means I necessarily have to shift to big picture guff and stop spending time on the details pieces. In short, I don't really get to engineer as much as before.

Of course, I'm an engineer at heart so the idea of ignoring the details is anathema and it frustrates me. When I'm deep in whiteboards, board meetings and shitty powerpoint at work, the side projects and home lab become my release valve. It's a coping mechanism, I think.

Likewise, when I'm sucked into low-level grind at work which happens from time to time, side projects stagnate. I don't need them for a while.

Re:The art of over-engineering your side projects by TeMPOraL on Sunday September 03, 2017 @03:46AM (#15160177)
Or they're bored out of their minds and trying to make their work not be so much of a waste of time.

Most of the stuff I've seen done at typical development companies are intellectually interesting maybe to people who really started learning to code on the job, and generally don't have much experience. They're discovering new things every day, so it's fun! However, if someone has already few languages and various types of projects behind them, then the job becomes much less technically challenging and much more repetitive.

Re:The art of over-engineering your side projects by patrickmclaren on Friday September 01, 2017 @12:35PM (#15150016)
IMO, there's room for both approaches, perhaps with the exclusion of the most the irrational situations as you describe. It's unreasonable to expect engineers to bring battle-tested solutions to the table 100% of the time.
Re:The art of over-engineering your side projects by devmunchies on Friday September 01, 2017 @04:47PM (#15152265)
I have a side project where I'm using the new Crystal language and even wrote a post on my company blog about it but we don't actually use it, nor do I try to get us to use it.
Re:The art of over-engineering your side projects by mrisoli on Friday September 01, 2017 @08:45AM (#15147881)
Me too, I think there are two types of side projects, this article focuses on those which I believe people will try to generate a business out of, or at least some form of passive income.

But there are also some side projects where you just want to get your hands dirty with some different tech, if you do deliver something then you can showcase it on your portfolio. I know when the app store gold rush first started a lot of people did this as experience to get iOs dev jobs.

Re:The art of over-engineering your side projects by dcminter on Friday September 01, 2017 @08:54AM (#15147943)
Exactly - I enjoy side projects precisely because they're not bound by pragmatic concerns. I can delve as deep as I like into getting the CI pipeline working just right, making sure the servers and cacheing are set up ok and all those other things that I usually have to leave to colleagues.

More often than not the skills I learn doing this kind of thing end up being very useful to me in the day job.

I'd like it if the side project was successful enough to justify all of that yak shaving, but I don't really expect it to be and I'm not that fussed when it isn't.

But if your primary goal is that your side project should become your day job then the article is pretty good advice. Leave your yak unshaven :)

Re:The art of over-engineering your side projects by TeMPOraL on Sunday September 03, 2017 @03:48AM (#15160185)
> they're not bound by pragmatic concerns.

99% of the time, if you want to go pragmatic, you should buy a ready-made solution (and then possibly adapt it to your use case). It's efficient (especially if you need the problem solved to make money) - but it's no fun.

Re:The art of over-engineering your side projects by mgolawala on Friday September 01, 2017 @11:15AM (#15149080)
I agree, but there are good reasons why it is often good to combine both types.

Considering this is only a side project, you have a high likelihood of abandoning it out of boredom, lack of traction or simply burn out. However, if your side project is "interesting" to you because you are experimenting with various tech, you come back to it because it is fun and it will keep you motivated.

It is totally cool to build something to learn and throwaway, but why not build something to learn that might also generate some income in the future?

Re:The art of over-engineering your side projects by bhouston on Friday September 01, 2017 @09:21AM (#15148115)
I learned how not to over-engineer specifically from learning from my mistake of over-engineering my side projects. :)
Re:The art of over-engineering your side projects by corobo on Friday September 01, 2017 @10:21AM (#15148512)
Nailed it with this comment. My side projects are for learning. There was a moment or two where I was aiming to monetise but I'm not at that stage and I now realise that, I just want to play with technology.

Actually starting to livestream them now as I figure if others can see what I'm trying to do I might be able to get instant feedback on how to do things better - crowd sourced learning :)

Re:The art of over-engineering your side projects by bpicolo on Friday September 01, 2017 @08:49AM (#15147905)
> Over-architecting infrastructure

That's not a problem, that's the goal!

Re:The art of over-engineering your side projects by amyjess on Friday September 01, 2017 @11:32AM (#15149300)
I have never regretted taking the time to make sure my projects have a proper logging infrastructure with every possible thing fed to the logger. I've built logging support libraries to set up AOP hooks to auto-log the call stack at all times, and it's been a lifesaver.

I can't imagine not over-architecting this kind of stuff. I know I'd be in a world of pain several times if I hadn't.

Re:The art of over-engineering your side projects by cpfohl on Friday September 01, 2017 @11:44AM (#15149440)
If you would have been in a world of pain, I'd argue you weren't over architecting. You were doing your job just right!
Re:The art of over-engineering your side projects by tokenizerrr on Friday September 01, 2017 @11:51AM (#15149532)
How do you prefer to store/ship logs?
Re:The art of over-engineering your side projects by eropple on Friday September 01, 2017 @01:57PM (#15150871)
Not 'amyjess, but the rightest answer I've found (and I do a lot of this stuff) is to use a Bunyan-style one-JSON-object-per-line log file and ship to something that can consume it (ELK, CloudWatch Logs, whatever). AWS CWL is dirt cheap and the awslogs daemon, while pretty far from perfect, does work pretty well. You can also couple it with CloudWatch Metrics.
Re:The art of over-engineering your side projects by marcosdumay on Friday September 01, 2017 @07:04PM (#15152996)
That's completely dependent on the format of your infrastructure. There is no answer that will fit everybody, and asking around for advice of strangers may even bias you in a damaging way.
Re:The art of over-engineering your side projects by tokenizerrr on Friday September 01, 2017 @07:28PM (#15153105)
Of course it is. I am familiar with all the common methods, but I'd like to hear how other people solved it for their specific situation.
Re:The art of over-engineering your side projects by overcast on Friday September 01, 2017 @10:19AM (#15148493)
This is exactly how I approach side projects as well. I use them as iterative learning experiences, with each new project building on the last to more complex ideas. Learn a TON that way, and potentially build something that people want to use. You can always go back, and add things you've learned down the line.

One thing I do not do, is just rebuild something someone else has already done in a new "stack", just to see if I can. Any project I undertake is a new idea I've not seen done, or with features I want.

Re:The art of over-engineering your side projects by rdiddly on Friday September 01, 2017 @11:29AM (#15149267)
Agreed; in fact, the title falsely (or maybe ironically) conveys a positive attitude about how it's an "art," thereby attracting people who, like me, wanted to hear about some gloriously and baroquely over-engineered side project that will never see the light of day. A large investment of time, effort and craft into something, for its own sake, with no heed to whether it will ever "ship": That's what art is, man!

Should've been titled something like "Don't Over-Engineer Your Side Projects" or "How to Make your Side Project Resemble Your Day Job (If You Work for Sensible People)"

Re:The art of over-engineering your side projects by shangxiao on Friday September 01, 2017 @10:06AM (#15148396)
I often explore new stacks by building a product that already exists complete with identical css! For example: Wanna try out aiohttp, vue & css grids? Let's build a Slack clone!
Re:The art of over-engineering your side projects by brightball on Friday September 01, 2017 @10:41AM (#15148682)
Yep. That's why I've rebuilt my little blog about a dozen times. I've built CMS's from scratch before (actually built a white label CMS generator) and it's a non-complicated process that I know deeply enough to push the pain points when I try things out.
Re:The art of over-engineering your side projects by Clubber on Friday September 01, 2017 @08:49AM (#15147904)
I do that too, but make that my secondary goal. If it fails, at least I got to improve my skills on whatever technology I chose to build it in. It helps me push along when it gets to the dull parts.
Re:The art of over-engineering your side projects by PaulHoule on Friday September 01, 2017 @12:45PM (#15150128)
When you are doing a side project you typically have a number of risk factors such as: (A) using new tools, (B) working in a new area, (C) building a new type of application, (D) uncertain marketing, etc.

Any kind of project has a "risk budget". If you have few other risks, then you can spend your risk budget on new tools. If you are facing high risks in other directions, it makes sense to stick with what you know.

Re:The art of over-engineering your side projects by meddlepal on Friday September 01, 2017 @09:48AM (#15148271)
Yea a lot of my side projects are started out with the understanding they're not going to become anything more than toys for me to play with ideas.

This article feels like it's written from the perspective of using a side-project to bootstrap a business rather than as a learning experience, hobby or toy.

Re:The art of over-engineering your side projects by ukd1 on Friday September 01, 2017 @12:36PM (#15150046)
Boom. This. I wrote a response to this in a little more detail - https://rsmith.co/the-actual-art-of-over-engineering-your-si...
Re:The art of over-engineering your side projects by adamkruszewski on Friday September 01, 2017 @02:17PM (#15151067)
Most of my side projects are like that -- just an excuse to learn something and more often than not abandoned when I have learned what I wanted. Not the best way to develop software but a great one to learn new technology.
Re:The art of over-engineering your side projects by cagmz on Friday September 01, 2017 @10:45AM (#15148724)
Where can one learn about different deployment approaches?
Re:The art of over-engineering your side projects by caleblloyd on Friday September 01, 2017 @11:20AM (#15149137)
By "deployment approaches", I mainly meant CI/CD and infrastructure. So Jenkins, Travis CI, etc. to do the build work and testing, then automatically push to Kubernetes, Docker Swarm, AWS Lambda, etc. on the infrastructure side.

For a side project, pick one on that you're interested in on each side. For example, maybe try out CircleCI and deploy to Google App Engine. Then you'll learn two new things, maybe one is great but the other is a pain in the ass. You can take the one that works well to a "real" project and talk knowledgeably to the architecture.

Re:The art of over-engineering your side projects by yingxie3 on Saturday September 02, 2017 @01:56AM (#15154544)
I on the other hand, try to convince my employer to take on the new stacks, so I can play with it while getting paid