Re:Running three hours of Ruby tests in under three minutes
by heywire on Thursday August 13, 2015 @04:53PM
(#10057141)
>Back-testing is also incredibly valuable: We have repositories of every conversation or transaction we've ever exchanged with the banking networks, and when making changes to parsers or interpreters, we can compare their output against the old version on all of that historical data.
Are you referring to test data or actual live transaction data? The latter would seem like a huge liability and target for hackers.
Re:Running three hours of Ruby tests in under three minutes
by nelhage on Thursday August 13, 2015 @05:16PM
(#10057257)
Live data, but they're stored redacted, and/or with sensitive data (e.g. credit card numbers) replaced with opaque tokens that reference an encrypted store that's carefully access-controlled.
Re:Running three hours of Ruby tests in under three minutes
by ersii on Friday August 14, 2015 @04:55AM
(#10059197)
Do you have any policy or decision made on how long you plan on storing that data? What I'm wondering, are the transactions currently "stored indefinitely"? (I'm referring to both data stores. The tokenized and the encrypted one)
Re:Running three hours of Ruby tests in under three minutes
by kevan on Friday August 14, 2015 @12:39AM
(#10058653)
> ranging from just keeping hard-coded strings containing the expected output, up to and including implementing our own fake versions of external infrastructure.
This sounds very familiar, we rely on external credit systems pretty heavily. We started by mocking service responses and including the response XML in our unit tests. Now we have a service simulator that returns expected values and has record/playback capability. It's not ideal and responses get outdated occasionally but we haven't found a more elegant way to handle it yet.
We don't have a single answer we use for every system we work on, but we employ a few common patterns, ranging from just keeping hard-coded strings containing the expected output, up to and including implementing our own fake versions of external infrastructure. We have, for example, our own faked ISO-8583 [1] authorization service, which some of our tests run against to get a degree of end-to-end testing.
Back-testing is also incredibly valuable: We have repositories of every conversation or transaction we've ever exchanged with the banking networks, and when making changes to parsers or interpreters, we can compare their output against the old version on all of that historical data.
[1] https://en.wikipedia.org/wiki/ISO_8583