News for hackers. Stuff that matters.
Re:The art of over-engineering your side projects by czep on Friday September 01, 2017 @02:21PM (#15151107)
The whole point of a side project is to over-engineer it! The thought of holding myself to deadlines and driving myself to "ship" is totally antithetical to the entire purpose of a hobby project. It's a welcome relief from the 9-to-5 constant necessity to cut corners and finish things without ever properly understanding them because there's no time.

For my side projects, I actually want to go down rabbit holes. I will try the same thing 7 different ways and benchmark the hell out of them. I'll read all the documentation and every blog post I can find. Learn a new language just to see if its features make things ever so slightly easier. Learn a new framework, write a new framework, throw it away and write a new one.

The point isn't to ship code, it's to learn. Honestly I don't care if it ever ships. And even when I do release one, I don't care if nobody uses it except me.

Re:The art of over-engineering your side projects by johnfn on Friday September 01, 2017 @03:32PM (#15151713)
HN is made up of two groups in ever constant warfare, the yin and yang, as it were- the group who thinks that programming should be fun, experimental and enjoyable, and the group that thinks that programming is a means to the end of getting a product to market and nothing more.

Whenever a post from one group (such as the article) comes out, a post from the other (such as OP) will quickly appear. And everyone loves it :)

For what it's worth, I think that both perspectives are valid. It depends only on your goals.

Re:The art of over-engineering your side projects by iamnafets on Friday September 01, 2017 @04:12PM (#15152024)
This is mostly off-topic, but being able to identify or root out the value mismatches between people in a discussion (as you've done) seems like an essential ingredient to productive discussion.

I've noticed that people who can do this are disproportionately better at achieving their goals than those who can't, especially in engineering organizations. Exceptions of course for zealots and revolutionaries.

Re:The art of over-engineering your side projects by rak00n on Saturday September 02, 2017 @02:03AM (#15154566)
"Observe, this is the beginning of philosophy, a perception of the disagreement of men with one another, and an inquiry into the cause of disagreement, and a condemnation and distrust of that which only `seems', and a certain investigation of that which `seems' whether it `seems' rightly..." - Epictetus, The Discourses.
Re:The art of over-engineering your side projects by chris_st on Saturday September 02, 2017 @09:00AM (#15155752)
Absolutely. This even works at a somewhat lower level... I remember hearing an argument between two cow-orkers, and realizing that each was thinking different (but similar!) things, but assuming the other was thinking the same thing. Angriness ensued...
Re:The art of over-engineering your side projects by Chris2048 on Monday September 04, 2017 @10:06AM (#15168278)
> Angriness ensued...

This usually only happens when one begins to make personal insults, i.e. "only a poor programmer would think that!"..

Re:The art of over-engineering your side projects by tenkabuto on Saturday September 02, 2017 @02:03PM (#15157360)
This skill is one of the key differences that I've noticed among philosophy students (majors) and everyone else, with their skill being far more developed than others'.

I'm not sure if persons with the skill are more disposed towards studying philosophy or the study of philosophy prompts the development and demonstration of that skill more, but hang out with the philosophy kids and you might come to refine this skill by social osmosis.

Re:The art of over-engineering your side projects by superjisan on Friday September 01, 2017 @04:29PM (#15152147)
Wow! This made me think deep thoughts man.
Re:The art of over-engineering your side projects by deanCommie on Saturday September 02, 2017 @06:02AM (#15155177)
The yin and yang becomes more amusing when you think that the site's title suggests an appeal to the first group: the techies for whom this is a passion.

Yet it would not exist without the second group because it was started by an investment fund/startup incubator.

I agree with you, I love both perspectives and the passions they ignite. I am squarely in the first group, and I love reading about the real life realities of product design/management, pitching, etc.

Re:The art of over-engineering your side projects by zippergz on Friday September 01, 2017 @05:05PM (#15152375)
I don't think these are mutually exclusive. At work, my goal is to create products that people use (and pay for). At home, my goal is to learn and have fun. Different approaches for different contexts.
Re:The art of over-engineering your side projects by danielharrison on Friday September 01, 2017 @06:24PM (#15152825)
This is not specific to programming, but personalities in general. There are people who love the journey and others who love the destination.

People rarely switch from one to the other. It's part of our core ideals.

However, there is a large, vocal, percentage from both these groups that will always voice their opinion against an opposition opinion. It's not a competition and there's no correct answer. It's purely subjective.

Re:The art of over-engineering your side projects by ramshanker on Friday September 01, 2017 @10:33PM (#15153877)
The quote. "We agree to disagree".

Or else.

War begins!

Re:The art of over-engineering your side projects by raldu on Friday September 01, 2017 @06:25PM (#15152832)
I think its embedded into the "maker" mindset in general and does not necessarily have to be one way or the other. It's both.

A writer can have lots of poetry, personal expository essays, journals, unfinished drafts lying around OR s/he can "ship" a best-selling novel, publish a high-traffic blog or something.

Similarly, a painter would experiment with lots of techniques, colors, textures "on the side," again having lots of personal work lying around, yet would probably treat something s/he would put in an exhibition differently.

In the first case, you are experimenting, discovering, breathing; while in the second case you have to think hard about your audience, your readers, or your customers.

It's built-in.

Re:The art of over-engineering your side projects by MaysonL on Friday September 01, 2017 @07:13PM (#15153041)
A writer can have lots of poetry, personal expository essays, journals, unfinished drafts lying around OR s/he can "ship" a best-selling novel, publish a high-traffic blog or something.

There are also writers who can do all of these: my personal favourite is Robert Graves (possibly because once many years ago I got him to autograph his Selected Poems and his memoir Goodbye to all That).

Re:The art of over-engineering your side projects by soheil on Friday September 01, 2017 @05:19PM (#15152456)
And then there is everyone else in between, including me who both want programming to be fun and enjoyable and on the other hand simultaneously care about getting a product to market.
Re:The art of over-engineering your side projects by howlgram on Saturday September 02, 2017 @06:03AM (#15155180)
yeah but those don't make for interesting articles or discussions
Re:The art of over-engineering your side projects by brianpan on Saturday September 02, 2017 @11:34AM (#15156476)
There are two groups of people on HN: people that think everything is binary and people that think everything is a continuum.

I'm somewhere in the middle.

Re:The art of over-engineering your side projects by jeffdavis on Friday September 01, 2017 @04:20PM (#15152072)
It's the same group of people, they just may be in different moods at different times.
Re:The art of over-engineering your side projects by abalashov on Friday September 01, 2017 @10:48PM (#15153935)
I was going to say the same thing. I've frequently argued both for the ruthless pragmatism of just-get-it-shipped and for self-indulgent exploration.
Re:The art of over-engineering your side projects by LrnByTeach on Saturday September 02, 2017 @04:04PM (#15157916)
Wow!!! this captures the essence of "Two schools of Thought" on Programming, the YIN and the YANG !!!!

> HN is made up of two groups in ever constant warfare, the yin and yang, as it were-

> the group who thinks that programming should be fun, experimental and enjoyable, and

> the group that thinks that programming is a means to the end of getting a product to market and nothing more.

> For what it's worth, I think that both perspectives are valid. It depends only on your goals.

Re:The art of over-engineering your side projects by edwinyzh on Saturday September 02, 2017 @12:25AM (#15154261)
There are things in-between, too - Enjoy the fun software development process to make an end product successful! And this is exactly my own feeling and goal, though I'm not doing it very well as of now ;)
Re:The art of over-engineering your side projects by paulmd on Friday September 01, 2017 @04:02PM (#15151950)
Yup, that's the yin and yang of programming. Programming is an art, but real artists ship.
Re:The art of over-engineering your side projects by D_Guidi on Friday September 01, 2017 @04:09PM (#15151990)
Yep, but artists don't ship every single portrait their made. Often, they experiment and they made practice
Re:The art of over-engineering your side projects by etherealG on Saturday September 02, 2017 @01:06AM (#15154386)
The yin and yang. I love it. :)
Re:The art of over-engineering your side projects by tomerbd on Saturday September 02, 2017 @04:04AM (#15154879)
WOW :)
Re:The art of over-engineering your side projects by anyfoo on Friday September 01, 2017 @08:31PM (#15153419)
Yes! My last hobby project (I explicitly point out "hobby" project instead of "side revenue", but the article mentions "hobby" in the first sentence as well) was exactly that.

Because it was a hobby project, I went completely mad on it. I built my own SPDIF interface, from scratch, using an FPGA. I let the hardware broadcast the sound frames over Ethernet. I also re-implemented Sony's proprietary consumer electronics control protocol, in hardware as well (a microcontroller could have easily done that instead). I greatly overengineered a circuit board, despite having a functional "test setup" already that did the job more than well enough. I wrote largely unnecessary client software, using language extensions I've never used before.

The task at hand could have been solved with buying a cheap USB sound adapter from eBay and some manual work instead. Had I gone down that route, I probably would have completed the task in a fraction of the time that building this project took.

But I had so much fun along the way and learned so much about electronics, FPGAs, SoCs, static typing for python etc., all because it was a project for me and nobody else.

One big lesson I've learned, though, is that having an actual task you want to solve greatly helps you in finishing your project. I ran out of steam a number of times and the project fell to the wayside, but because the underlying task was not fulfilled yet, I always went back to it and quickly got completely focused on it again. Projects that I only did for playing around and learning most often got abandoned.

Re:The art of over-engineering your side projects by krptos on Friday September 01, 2017 @02:41PM (#15151315)
Agreed. Generally, at any point of time I have two types of side projects going on:

#1. One that I am very clear about how to implement. One that I want build, ship and grab users as fast as I can. The real goal of the project is to get it out their in the market and make a tiny dent in the universe ;)

#2. One that I am not very clear about what it's gonna become. This one I want to take it slow. Experiment. Play with tech that I haven't had a chance to flirt with. The goal is to enhance my toolbelt and learn new concepts/tech.

Clearly keeping these projects separate helps me keep things clear. I quickly take decisions and move fast with one. I play and experiment a lot, no matter what the outcome is, with the other one.

Re:The art of over-engineering your side projects by collinmanderson on Friday September 01, 2017 @03:48PM (#15151850)
Exactly. Side projects are often the best time to try out new tech. It's low risk.
Re:The art of over-engineering your side projects by imhoguy on Friday September 01, 2017 @04:09PM (#15151994)
There is also the third way -not to earn, not to necessarily learn, but simply to feel pleasure from making a software art and experimenting with things. When playing with my side project I feel like a painter adding new lines to the same canvas for years, like Gaudi spending his 15 years of life constantly redesigning Sagrada Familia. Well, I also learn stuff this way but often I get bored with them once challenge is completed ;)
Re:The art of over-engineering your side projects by 0xfaded on Friday September 01, 2017 @03:57PM (#15151917)
Totally agree. I had a problem which required finding the minimum distance between an ellipse and a point. If that had been my day job I would have settled for Newton's method and hacked around the regions of bad convergence. Instead I spent a week developing to be what I still believe to be a novel numerical method that is much more stable over all regions.

Turns out I was wrong in some earlier maths, and the curve I wanted to find the distance to wasn't an ellipse. I was able to generalise the method for arbitrary curves, and now I've solved my problem!

Relevant blog post: http://wet-robots.ghost.io/simple-method-for-distance-to-ell...

Re:The art of over-engineering your side projects by delhanty on Friday September 01, 2017 @05:31PM (#15152539)
>I had a problem which required finding the minimum distance between an ellipse and a point. If that had been my day job ...

OT: I had the problem as part of my 1st day job in 1990 on Parasolid.

IIRC, there was more than one way of converting the problem into solving a quartic polynomial, which is solvable by radicals and so doesn't need Newton's method.

If the point is inside the evolute of the ellipse then the quartic has four real solutions, which gives two local minima and two local maxima of distance.

If the point is outside or on the evolute of the ellipse then the quartic has two real solutions, which gives one local minima and one local maxima of distance.

Having said that, all methods using quartic polynomials could be unstable if the ellipse was pathological, for example very long and thin. Maybe that would be a time to revert to Newton's method.

Re:The art of over-engineering your side projects by 0xfaded on Friday September 01, 2017 @05:55PM (#15152688)
There are certainly multiple ways of doing it. I'm curious, do you remember if the specific quartic had a nice "trick" to the formulation? I played with it a little and couldn't see anything that would cancel in a nice way.

I try to avoid using quadratic solutions in code because of numeric instability issues. Granted I never implemented the direct solution, but the number of opportunities for catastrophic cancellation gave me shivers.

Re:The art of over-engineering your side projects by delhanty on Friday September 01, 2017 @07:15PM (#15153048)
I don't think there was any trick.

One approach is to use the rational quadratic parametrization of the ellipse and substitute that into the squared distance function to get a quartic.

>I try to avoid using quadratic solutions in code because of numeric instability issues.

Sensible!

Back then the quartic solver had been written already and extensively tested by someone else - probably for line X torus intersection.

Even so there were numerical problems introduced because the quartic was in power basis and so the coefficients didn't have geometric meaning. I guess phenomena like Wilkinson's polynomial could occur. [0]

If I was doing it today, I would probably proceed as follows:

1. special case for the point exactly on an axis.

2. otherwise, choose the quadrant containing the point and represent that as a rational quadratic Bezier curve. This parametrizes the quadrant from 0 to 1 and is guaranteed to contain the global minimum.

3. substitute the quadratic Bezier curve into the square distance function to get a Bernstein basis quartic polynomial.

4. differentiate to get a Bernstein basic cubic.

5. solve the roots of the cubic numerically in the range 0 to 1: either Newton's or clipping or some other hybrid numerical method. A good reference is [1].

[0] https://en.wikipedia.org/wiki/Wilkinson%27s_polynomial

[1] Shape Interrogation for Computer Aided Design and Manufacture by Nichola M. Patrikalakis and Takashi Maekawa

Re:The art of over-engineering your side projects by mkl on Friday September 01, 2017 @07:41PM (#15153176)
> substitute that into the squared distance function

How the heck did "" get in there?

> 5. solve the roots of the cubic numerically in the range 0 to 1: either Newton's or clipping or some other hybrid numerical method.

Is there a reason not to use the cubic formula in this situation?

Re:The art of over-engineering your side projects by delhanty on Friday September 01, 2017 @07:49PM (#15153218)
>How the heck did "" get in there?

Something like

   (E(t)-P).(E(t)-P)
where E is the ellipse and P is the point.

>Is there a reason not to use the cubic formula in this situation?

Numerical stability in edge cases, which will occur for code exercised as extensively as Parasolid's.

I guess you could a Bernstein version of the cubic formula, though I've not seen one derived in the wild. It would be nice to have as well as Bernstein quartic formula. The Bernstein quadratic is easy.

Re:The art of over-engineering your side projects by scottlocklin on Friday September 01, 2017 @04:38PM (#15152200)
Can't you just use vector math for this?
Re:The art of over-engineering your side projects by Scea91 on Monday September 04, 2017 @03:40AM (#15166433)
Yes, vector math with a bit of number math and symbol math should do the trick.
Re:The art of over-engineering your side projects by roberttod on Friday September 01, 2017 @05:25PM (#15152497)
> The point isn't to ship code, it's to learn. Honestly I don't care if it ever ships. And even when I do release one, I don't care if nobody uses it except me.

The objectives for a side project are subjective, and so they should be. However I doubt most people truly have no intention to ever ship a side project or "don't care if nobody uses it", as admirable as that would be.

I look back at my naive, recently graduated self and remember how I truly wanted to ship my side projects but loved to play around with new toys so much that I rarely shipped anything. I wish I had read a few more articles like this that would have helped me strike a balance between trying new things but still having enough experience in the stack I'm using to actually ship.

In short, if you have a great idea you want to ship then deprioritise learning to stand the best chance of getting it out there!

Re:The art of over-engineering your side projects by pcmaffey on Friday September 01, 2017 @11:40PM (#15154105)
I'm pretty sure you just made an objective subjective about subjective objectives.
Re:The art of over-engineering your side projects by andmarios on Friday September 01, 2017 @02:26PM (#15151159)
Totally agree. The author seems to write about a specific subset of side projects: SaaS with a possibility to commercialize.
Re:The art of over-engineering your side projects by ben_jones on Friday September 01, 2017 @02:42PM (#15151330)
I think there's an important conflation of what a side project is to many people. If I mention to any of my non technical friends that I'm building an app their minds go to the movie "The Social Network" or to Steve Jobs or Bill Gates. They become excited and interested - up until the point where they realize it has nothing to do with Facebook or Tinder or Instagram.

I have no doubt that this effects the side projects developers choose, or how they present them. Places like HN will always to a certain extent push people to "launch" and monetize their projects.

Re:The art of over-engineering your side projects by turboladen on Saturday September 02, 2017 @02:42AM (#15154667)
Having done this a number of times with side projects, I can say I both agree and disagree. I've learned so much from delving down black holes: new languages, frameworks, design patterns, tools, etc.--stuff that stimulates my brain and helps me feel like I'm becoming a better engineer. ...but not realizing an idea in my head for a tool that I just want to use because I can't find it anywhere else is really annoying. I kinda hesitate to even start projects anymore because I fear I'll spend hours and hours, burn out, then end up with nothing I can use.
Re:The art of over-engineering your side projects by bleair on Sunday September 03, 2017 @01:19AM (#15159812)
Agree fully. Personal projects let you experiment, they let you try things out to see just how hard or easy building that set of features you were contemplating is.

Now, if you personal project is intended as a gateway to your bootstrapping of commercial venture you _must_ make completely different choices in what/how you build (lean, and minimum useful/viable project) tec.

Re:The art of over-engineering your side projects by drakonka on Saturday September 02, 2017 @03:16AM (#15154753)
I agree; I've been working on my mess of a side project for years now. I let the scope grow tremendously, and started over more times than I can remember. It's fun to experiment with potentially better or more interesting ways of doing something in a side project without worrying how much time it "wastes" in the long run.
Re:The art of over-engineering your side projects by drdrey on Friday September 01, 2017 @07:11PM (#15153029)
The premise of the story is that you want to deliver an MVP. Your premise is that side-projects are a nice way to geek out on tech. Both are valid, but you have to make sure what you're trying to accomplish before spending quality time with rabbit holes.
Re:The art of over-engineering your side projects by jacalata on Saturday September 02, 2017 @04:54AM (#15155010)
> The point isn't to ship code, it's to learn.

The main reason I've wanted other people to use the things I've made is to learn more about what I've built and how it could be better.

Re:The art of over-engineering your side projects by iteriteratedone on Friday September 01, 2017 @06:49PM (#15152926)
I offer a middle path , a bhudist interpertation of the problem. A project can be both fun/innovative and productive. You iterate on the fun , then you iterate on the productive. Youll never acheive completeness in both. Youll always suffer from needing to improve on aspect. Life is suffering , learn to accept and not let that reality stop you from creating.

But some projects are more innovative or productive than others. Maybe you should abstract those projects away from eachother? You could put them in seperate classes. You could have the productive project use the innovative project.