Re:The accidental HFT firm
by kbenson on Wednesday January 24, 2018 @11:56AM
(#16224174)
It's not always about a hook, sometimes it's about where yo break into subsystems and subfunctions. Breaking it up at the right point allows for easy extension through writing new components later, while requiring the least amount of modifications to the production code (and thus hopefully less bugs introduced into working code-paths). Additionally, how you represent your data may allow for easier extension of the data type or additional, related data type processing later.
I do a lot of data mining, and the first step is always securing the data source and archiving it. There have been times where I wasn't able to revisit the project and finish the actual parser of the data for 5-6 months, at which point I can run through the old data. Other times, I'll write the parser but we'll only be parsing a subset of the data we acqiured. The additional unparsed portionisn't necessarily without merit, it's just not something we are dealing with yet (either because it's still of an unknown quality, or requires more effort than we want to spend at that moment). Knowing it's there and that it may need to be parsed in the future can subtly affect your design such that you leave yourself the openings you'll later need to easily parse this additional data. Sometimes that's as simple as just avoiding situations that would make it extremely cumbersome later, and not necessarily actively making it easier.
I'm not sure ignoring something as obvious as this makes one a good developer, but what's "obvious" and what isn't is actually quite different at different software engineering experience levels.
I do a lot of data mining, and the first step is always securing the data source and archiving it. There have been times where I wasn't able to revisit the project and finish the actual parser of the data for 5-6 months, at which point I can run through the old data. Other times, I'll write the parser but we'll only be parsing a subset of the data we acqiured. The additional unparsed portionisn't necessarily without merit, it's just not something we are dealing with yet (either because it's still of an unknown quality, or requires more effort than we want to spend at that moment). Knowing it's there and that it may need to be parsed in the future can subtly affect your design such that you leave yourself the openings you'll later need to easily parse this additional data. Sometimes that's as simple as just avoiding situations that would make it extremely cumbersome later, and not necessarily actively making it easier.
I'm not sure ignoring something as obvious as this makes one a good developer, but what's "obvious" and what isn't is actually quite different at different software engineering experience levels.