Re:You don’t need to be “enterprise-ready” or “scalable”
by gumby on Tuesday May 31, 2022 @07:22PM
(#31576127)
My gf works in UX for a major bank and they simply issued her a bank account in her name (with her SSN etc -- it's a real account, with ATM and debit cards and the like). If she and some co workers want to test something out relating to accounts they actually use these accounts to minimize this kind of problem.
They must have some special code that says that these accounts can have their fees zeroed and probably are subject to some special out of band audits. I don't really know the details of course but I was actually kind of impressed.
(no she is not actually a customer of her employer)
Re:You don’t need to be “enterprise-ready” or “scalable”
by tixocloud on Tuesday May 31, 2022 @09:26PM
(#31576996)
Previously led data teams in banks and you’re right. There are dozens of flags for test accounts. When auditing, we screen out these accounts. Of course, with a proper data cleaning process, analytics gets a lot easier
Re:You don’t need to be “enterprise-ready” or “scalable”
by lobocinza on Wednesday June 01, 2022 @02:11AM
(#31578710)
That reminds me of "Mostly Harmless" from Douglas Adams where Ford Prefect breaks into The Guide's offices and gets himself an infinite expense account from the computer system. That's a dangerous thing to have.
Re:You don’t need to be “enterprise-ready” or “scalable”
by SheinhardtWigCo on Wednesday June 01, 2022 @02:27AM
(#31578816)
Problem is, anyone with a good story to tell should really keep their mouth shut.
Re:You don’t need to be “enterprise-ready” or “scalable”
by invalidname on Tuesday May 31, 2022 @10:09PM
(#31577269)
I worked at banks too but mostly segregated from the core business. Today I see observability tools as the solution for problems like that. Specifically tools like Lightrun where you can observe production, mitigate and fix directly from the IDE.
I wonder if other developers are familiar with this approach?
Re:You don’t need to be “enterprise-ready” or “scalable”
by shardullavekar on Wednesday June 01, 2022 @05:30AM
(#31579727)
We are working on Videobug - it records code execution on production and plays in the developer IDE. Our Show HN here: https://news.ycombinator.com/item?id=31286126
Re:You don’t need to be “enterprise-ready” or “scalable”
by acwan93 on Tuesday May 31, 2022 @05:59PM
(#31575455)
Same here in e-commerce. We end up testing in production environments with an element somewhere that's in a development environment that can be tracked and actively watched. Amazon notoriously does not have a production environment, and eBay's is just unusable.
Re:You don’t need to be “enterprise-ready” or “scalable”
by grogenaut on Tuesday May 31, 2022 @06:51PM
(#31575874)
im pretty sure amazon has a production environment... just their staging is problematic. It's incredibly expensive to run a copy of prod and keep it up. You essentially have to get paged for it doubling your burden.
Re:You don’t need to be “enterprise-ready” or “scalable”
by acwan93 on Tuesday May 31, 2022 @08:27PM
(#31576590)
Yep, I mistyped. Amazon MWS doesn't have a staging/sandbox environment that's available for third party developers to test on.
Re:You don’t need to be “enterprise-ready” or “scalable”
by nijave on Tuesday May 31, 2022 @05:51PM
(#31575392)
Worked with banking software before; it can be incredibly complicated. I worked on proof of concept infrastructure for a Websphere monolith that ran on AIX servers. It had at least 10-15 services it connected to and each one of those were huge Java monoliths, too. Some of those apps had 8+ copies of /each/ environment (8x dev, 8x test, 4-8x stage/uat, 1-2x prod) so you could do integrated tests without stepping on other people or something software (software B might have a dedicated test environment of software A)
Re:You don’t need to be “enterprise-ready” or “scalable”
by halfmatthalfcat on Tuesday May 31, 2022 @06:40PM
(#31575795)
I haven't found a wholistic integration testing framework where you can effortlessly mock out upstreams as running services, so that you don't have to ham-fist integration testing code into your app or rely on actual, live upstreams which are probably prone to breakage anyway.
Re:You don’t need to be “enterprise-ready” or “scalable”
by nijave on Tuesday May 31, 2022 @09:25PM
(#31576991)
One way to balance is contract testing where you run a test suite against your mock/stub and the same test suite against the real thing. Your mock/stub would usually just have hardcoded data that aligns with the test suite to pass.
Another way is creating separate code libraries to abstract away external services. For instance, you might write ObjectStorage library that has real integration tests that use S3 but in your application you just stub out the ObjectStorage library. A lot of frameworks have verifying doubles or "magic mocks" or similar that will stub the method signatures so that can help make sure you're calling methods correctly (but wouldn't help catch data/logic issues).
Live upstreams are also prone to inconsistent state and race conditions--especially when you try to run more than 1 instance of the test suite at once.
For web services, I also like using "fixtures". You just manually run against a real service and copy the requests to a file (removing any sensitive data) than have your stub load the data from the file and return it. If you hit any bugs/regressions, you can just copy the results that caused them into a second file and add another test case.
Re:You don’t need to be “enterprise-ready” or “scalable”
by mieubrisse on Wednesday June 01, 2022 @12:59AM
(#31578281)
Shameless plug: this is exactly the spirit behind what we're building at Kurtosis! (https://www.kurtosistech.com/) You get an isolated environment running real services connected to each other, so if you have a container with your mock then from your app's perspective it will look just like the actual upstream.
Re:You don’t need to be “enterprise-ready” or “scalable”
by lifeisstillgood on Wednesday June 01, 2022 @05:17AM
(#31579666)
I have looked at your docs and don't quite get it. Is the idea that I have say 10 microservices that app-in-test will / may use, and then I set up the 10 instances (somehow all in one container?) and then can test that.
I think that seems sensible, but how is that different to 10 containers ? Is the issue something about simpler network management inside a container?
Re:You don’t need to be “enterprise-ready” or “scalable”
by nerdponx on Tuesday May 31, 2022 @09:06PM
(#31576868)
I've done things like this in ad-hoc setups with Docker Compose and environment variables, managed with shell scripts. I would be very interested in a more thoughtful alternative that was integrated with testing.
Re:You don’t need to be “enterprise-ready” or “scalable”
by chupasaurus on Wednesday June 01, 2022 @03:29AM
(#31579109)
I've done a thing automating whole environment setups with VMware cloning (undocumented feature because there is it's own product on top of it) and Docker via CI, in a bank. Basically we did golden images of VMs with databases having all accounts for test cases and prepared images for services to be deployed, while getting all necessary variables from CI and VSphere to write down in configs and Consul/Vault via scripts. Initial version control for the env could be done via same prefixes in the Git branches of services, after the deployment each env had a way to deploy whatever via CI's Web UI or API.
Running automated integration tests on the whole env is just an added step in the pipeline.
Re:You don’t need to be “enterprise-ready” or “scalable”
by raffraffraff on Wednesday June 01, 2022 @04:07AM
(#31579331)
It's funny, I worked in banking about 15 years ago and the technology stack was the same. Java monoliths, Websphere on AIX (plus mainframes naturally)
Re:You don’t need to be “enterprise-ready” or “scalable”
by winrid on Tuesday May 31, 2022 @06:02PM
(#31575481)
This works well. I've seen deployments where you have test1, 2, 3 etc, and you can "lock" the environment to yourself for a few hours.
Re:You don’t need to be “enterprise-ready” or “scalable”
by tluyben2 on Wednesday June 01, 2022 @02:47AM
(#31578907)
Yep, work in b2b banking as well and there is a huge issue with banking ‘test’ environments. Our partners (100s of companies, mostly banks/brokers/insurers/etc) mostly don’t even have test envs and if they do, they don’t represent production at all (sometimes the api is a completely different version etc). Our software allows to spin up a test/dev env locally or where ever but nothing much works without the partners and to dev/test those, those, you will have to dev/test against their production systems with real money.
In the beginning (more than 20 years ago), I insisted that we implement test systems ourselves so we didn’t have to test in production; that was naive; it was expensive and there is so much complexity that this does not work well.
Yes. I work in B2B banking software, so having an "authentic" environment to test against is extraordinarily complex. Even the big vendors in the space seem unable to maintain accurate facsimiles of production with their own decades-old core software, so smaller vendors stand no chance unless they adjust their entire strategy.
What we ultimately had to do was negotiate a way to test in the production environment in such a way that all the activity could be tracked and backed out afterwards. A small group of important customer stakeholders were given access to this pre production environment for purposes of validating things.
This is something that has never sat well with me. But, the way the business works the alternatives are usually worse (if you want to survive long-term, that is). We wasted 18 months waiting for a customer to have a test environment installed before we learned our lesson about how bad those environments are in reality.