News for hackers. Stuff that matters.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by belter on Saturday January 01, 2022 @07:20AM (#29757715)
Many eons ago...Around 10 to 15 years ago...Seen a similar serialization and date processing bug with a Java based Enterprise Integration Platform used by some of the largest companies in the world.

Several European Central Banks were unable to process transactions ;-). Can't provide more details, to protect the innocent. Since then, it has been a standard within the Testing group of that vendor to always have a running platform setup with a date 6 months ahead ( clock sync from a different source ). Something I added to my own Software Engineering and Testing standard practices.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by game_the0ry on Saturday January 01, 2022 @09:21AM (#29758452)
A couple of years ago, I did work for a major airline where the test suites that would test date processing would fail on every leap year (Feb 29). So they decided not to run tests or deploy on Feb 29th.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by jlkuester7 on Saturday January 01, 2022 @09:29AM (#29758518)
Lol, reminds me of of the large EMR platform I used to work on that (until very recently) could not handle the "fall back" DST event (thanks to local date encoding in very parts of the system).... So the recommended 'fix' for hospitals was to just turn it off (their entire electronic health record) for the duration of the DST changeover... I personally believe that a non-trivial amount of people die due to DST and similar time related bugs...
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by tharkun__ on Saturday January 01, 2022 @04:00PM (#29762721)
Same for SAP. Seems like it's still 'best practice' to do so:

http://www.sapbasis1solution.com/how-to-prepare-sap-system-f...

    During this time of one hour “Fall Back” on the clock, as is our Basis team best practice (SAP Note #7417, #102088 and others) we intend to gracefully shut down all Production applications prior to 2:00 AM, at which time the “fall back” happens that will revert back clock time by one hour.  We would then wait for clock time to advance past the “new” 2:00 AM Local time.  And start from 2:05 AM EST will bring up applications.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by curiousllama on Saturday January 01, 2022 @09:45AM (#29758647)
Oh, EMR systems kill patients all the time.

Like, I’m not convinced “turn it off” isn’t an improvement.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by game_the0ry on Saturday January 01, 2022 @10:16AM (#29759037)
I hesitate to find it amusing that healthcare software is so busted that we laugh off the negative, fatal consequences.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by tata71 on Saturday January 01, 2022 @12:23PM (#29760490)
No, the commenter isn't laughing or joking.

They're that bad.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by thfuran on Saturday January 01, 2022 @12:51PM (#29760772)
Not having one at all is also bad.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by throwawayboise on Saturday January 01, 2022 @01:28PM (#29761104)
How so? "First, do no harm" is the foundation of medicine.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by wereHamster on Saturday January 01, 2022 @02:04PM (#29761483)
Not of software engineering though.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by patmorgan23 on Saturday January 01, 2022 @06:37PM (#29764048)
Paper medical records systems cause errors/deaths for several similar reasons that electronic medical record systems. EMR's with all their faults are probably a net positive.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by dragonwriter on Saturday January 01, 2022 @03:46PM (#29762566)
> How so?

Empirically, it's associated with a higher rate of medical errors, which is a big part of the drive for electronic records.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by mr_toad on Saturday January 01, 2022 @09:23PM (#29765108)
Poor record keeping practices in general have been linked to killing patients. Especially ones involving medication errors. It’s not like paper systems were perfect or even demonstrably better.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by riahi on Saturday January 01, 2022 @10:24AM (#29759143)
From personal experience, this is Epic.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by belter on Saturday January 01, 2022 @12:10PM (#29760361)
For anybody who missed this very insightful comment :-) see the "Criticisms and controversies" section: https://en.wikipedia.org/wiki/Epic_Systems#Product_and_marke...
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by tuukkah on Saturday January 01, 2022 @05:38PM (#29763597)
Yes. Epic was deployed in Helsinki in 2021, and you bet it miscalculates the administration of recurring medications during DST changes, which nurses then need to correct manually.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by mark-r on Sunday January 02, 2022 @12:26PM (#29770892)
DST has long overstayed its usefulness. I wonder why it's so hard to get rid of?
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by Hamuko on Saturday January 01, 2022 @09:38AM (#29758579)
Our unit tests break twice a year, starting a couple of days before DST change. I'm sure it could be solved, but since the DST change happens on Sunday and the CI builds fail on Friday, no one has really bothered.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by Dunedan on Saturday January 01, 2022 @01:11PM (#29760963)
So I guess your units tests run with the current date and don't test behavior with different dates. That's something which has the potential to hide quite a few date related bugs until it's too late.

What I like to do to avoid this situation is to use property-based testing to test that the code works for all dates in the desired time span. For Python I use hypothesis [1] in combination with freezegun [2] for that. Here is an example of how that could look like:

  from datetime import datetime, timezone as tz
  
  from freezegun import freeze_time
  from hypothesis import given, strategies as st
  
  
  def business_logic():
      # TODO: Refactor before 2038!
      if datetime.now(tz=tz.utc) > datetime(2038, 1, 19, 3, 14, 7, tzinfo=tz.utc):
          raise Exception("This code is not working after UNIX time overflow")
  
      ...
  
  
  @given(dt=st.datetimes(max_value=datetime(9999, 12, 31, 23, 59, 59), timezones=st.just(tz.utc)))
  def test_business_logic(dt):
      with freeze_time(dt):
          business_logic()
  
  
  test_business_logic()
[1]: https://hypothesis.readthedocs.io/en/latest/

[2]: https://github.com/spulec/freezegun

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by Hamuko on Saturday January 01, 2022 @02:39PM (#29761859)
It's just that the unit tests are built on the assumption that if you go forwards 24/48/72 hours, the time will be the same but N day(s) ahead. And that breaks down when you shift time zones.

>don't test behavior with different dates

It basically tests with different dates every time it's run.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by denton-scratch on Saturday January 01, 2022 @01:49PM (#29761314)
> the CI builds fail on Friday

That seems different from what I understand by "continuous integration".

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by patmorgan23 on Saturday January 01, 2022 @06:39PM (#29764067)
It's just an unintentional read-only Friday
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by treeman79 on Saturday January 01, 2022 @01:23PM (#29761060)
We doing a midnight deploy with 40 people. After boss explained process. I asked him, so are we deploying on the first midnight or the second?

Long quiet pause, then a series of curses. So we deployed at 2am instead.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by mlyle on Saturday January 01, 2022 @02:43PM (#29761890)
Hmmm. Where are you? Here in the US DST changes happen at 2AM to avoid ambiguities/weirdness with midnight.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by rbanffy on Saturday January 01, 2022 @02:05PM (#29761486)
Midnight UTC solves most of these problems.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by mdavidn on Saturday January 01, 2022 @11:38AM (#29760014)
One project had tests fail every Saturday and Sunday, near the organization’s fiscal week boundary. Ever since then, I have strived to hard-code all dates in test suites, choosing dates several decades in the future.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by rbanffy on Saturday January 01, 2022 @02:06PM (#29761506)
The outcome of tests shouldn’t depend on the time of day. Are these integration tests and you are having to deal with external bugs you can’t fix?
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by alisonkisk on Saturday January 01, 2022 @12:20PM (#29760456)
Doesn't that just hide the bugs? Your tests should pass on any and all date.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by mdavidn on Tuesday January 04, 2022 @11:37AM (#29796707)
No. I write explicit tests on the boundaries, such that they would fail on any day of the week.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by karmakaze on Saturday January 01, 2022 @07:58AM (#29757932)
I came to ask if anyone was aware of systems that automate tests of dates in the future to find it already answered.

> Something I added to my own Software Engineering and Testing standard practices.

Noted.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by hughrr on Saturday January 01, 2022 @08:35AM (#29758135)
Isn’t this basically fuzzing?

Making sure it works at the business domain level is another thing entirely.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by matsemann on Saturday January 01, 2022 @08:56AM (#29758265)
Not exactly fuzzing, no.

Most tests are written with a fixed date, or now(). Instead, using now()+delta can help discover future bugs and give you some time to remedy the issue. Often annoying to write tests like that, though, without them breaking for unrelated/timing issues compared to a fixed value.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by chmod600 on Saturday January 01, 2022 @02:28PM (#29761745)
If you expect your software to exist in 6 months, then the input is entirely expected, and therefore not fuzzing.

It's a nice early-warning system for Y2K or Y2038-like problems.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by fragmede on Saturday January 01, 2022 @08:56AM (#29758266)
not especially. Fuzzing would be running the software with an API to test all of the possible dates.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by jacquesm on Saturday January 01, 2022 @09:20AM (#29758446)
No, that's exhaustive testing.

This is fuzzing: https://en.wikipedia.org/wiki/Fuzzing

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by azernik on Saturday January 01, 2022 @09:48AM (#29758667)
A date 6 months in the future is not invalid or off; it is a normal input that the program is expected to receive in the future.

Fuzzing would be testing the date parser by passing it '25-44-2021' to see what breaks.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by pimlottc on Saturday January 01, 2022 @10:00AM (#29758840)
Fuzzing inputs don’t necessarily have to be invalid, just dynamically generated in order to probe program states not normally exercised by manually generated test data. Per Wikipedia: “providing invalid, unexpected or random data as inputs”
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by hughrr on Saturday January 01, 2022 @10:22AM (#29759117)
Exactly my original point.

Invalid input is invalid input

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by 0x0000000 on Saturday January 01, 2022 @01:30PM (#29761128)
Valid, random inputs is still fuzzing, which is what the person you are replying to was sharing with the quote from Wikipedia.

Inputs don't have to be invalid for it to be fuzzing.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by jacquesm on Saturday January 01, 2022 @09:56AM (#29758768)
Indeed. But you probably meant to address the parent, rather than my comment.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by azernik on Saturday January 01, 2022 @10:11AM (#29758969)
Blerp
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by trembonator on Saturday January 01, 2022 @08:18AM (#29758037)
Cmon, it’s Microsoft! You shouldn’t need to do that!
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by jrootabega on Saturday January 01, 2022 @09:16AM (#29758407)
Agreed. Microsoft should be doing that already.
Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by nurettin on Monday January 03, 2022 @07:53AM (#29780264)
Reminds me how Homakov forward-tested github issues by sending a message from 3012 (it worked)

Here's the original link https://github.com/rails/rails/issues/5239

Unfortunately, they fixed the date and his bender account got renamed.

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by unbanned on Saturday January 01, 2022 @01:14PM (#29760986)
Is it someone's responsibility to periodically use the environment as real users?

Without this, there's no value in running a duplicate environment like this. Automated tests won't capture human interaction perfectly...

Re:Microsoft Exchange stops passing mail due to bug on 1/1/22 by rbanffy on Saturday January 01, 2022 @02:09PM (#29761529)
You can automate at the UI level and run that set of interactions (as well as screen captures and image matching if needed)