I've programmed for two decades. I write non-trivial software modules in a specific domain. It's not as if I was averse to analyzing things. And I try. When I can.

Yet. Design is damn hard.

One of the problems is that implementation is often an empirical process of requirements and constraints recovery.

I.e. I cannot create an elegant design before I've implemented anything, because some of the requirements are discovered only when the domain is explored using the code which I write.

One of the problems is that I'm not smart enough to design the system beforehand. And I don't have enough working memory to do so. And I presume many programmers work in similar environments.

I think in the 70's or 80's there was a vivid dream that one day programmers would have advanced CASE tools to help them design better programs. I wonder what happened to that dream. Maybe IDE:s and software systems became so easy to use that everyone prefers to prototype in code and perform empirical constraint discovery just like me...

If you got capable developers, let them do their job and create a flexible (expecting changes, simple design, usually not an elaborate many classes design) solution, that actually fits your situation. Developers are (should be) abstractions specialists. They should be able to find working ones in most cases.

Or live with limitations and the spread of thinking, that some things are not possible in various appartments. Mental barriers, that stop creative thinking, simply, because the tools do not exist.

In my experience, they are at most two of these things. Most of the time, I see lots of businesses bend over backwards to mold their processes to match the tooling they use, or expend lots of manual labour because their business flow isn't modelled efficiently in their tooling. Impedance mismatch is the #1 factor why many business automation projects end up costing multiples of what they were initially intended to save.

These problems aren't limited to off-the-shelf solutions, it could take an experienced developer months to really learn the business process and model it efficiently. Most businesses don't allow for that in their software budget, so you get either:

- something that works, and is relatively cheap, because it was cobbled together by someone from the business (see: excel sheets, access applications or nowadays, Microsoft Flow)

- something that works and is efficient, because it was built by a team of developers in close cooperation with the business

- something that's cheap and efficient but doesn't work, because it was built by a fly-by-night developer on a shoestring budget.

The only cases where you can have all three is when the business was built around a single piece of well-designed software, the CTO and CEO have been very conscientious about following the capabilities of the software, and the software manufacturer never decided to completely overhaul their interface for "reasons".

E.g. we have a bunch of stuff in Airtable. It's great. You can do amazing stuff with it, and it has a great API for most things.

Except for the change history and comments.

When you're aware of this, it is fine, but the moment your business processes start depending on using the comments and change history, you're locked in unless you're willing to use phantomjs or similar to log in and use the API (of course there is one) that their web client uses to access the comments, in a way that is a massive pain.

I'm happy for users to start building stuff and automating stuff. The bigger problem is when they don't design and don't understand the trade-offs. Often it works great for years, until it doesn't and you have a massive job untangling dependencies in some cobbled together solution.

Sometimes that is worth it.

Sometimes it causes you to incur costs far higher than what you saved initially because knowledge often gets lost as these system grow in complexity without any coherent thought behind them.

Very often, even if you end up using these tools, you'd still be far better off if there was still a review process to get someone to say "stop,if you do it this way you're creating a risky dependency, do it this way instead." Or to say "we really need the dev team to handle this."

I think that's a bit harsh - a lot of these tools can be pretty powerful if they are used for the kinds of tasks that they are well suited for. However, stray into a task that is just that bit more complex than what they are good for and they can be an utter nightmare.

Of course, deciding whether a particular task is a good fit for a particular tool can itself be a tricky question.

And as most of the business value lives between the UI and the database, I understand why there is no driving incentive to create similar high quality tools for numerical, data intensive programming tasks that run on the customers desktop. We work in a niche, hence must be self sufficient.

I've spent (in bizdev, e-gov) tons of time figuring out badly structured data, fixing data quality issues, handling tons of administrative tasks to access sensitive data, etc.

I think it's up to a point where the structure of the databases actually influence the structure of the government. That's because nowadays, the databases and the business rules they embody are so complex that even the civil servants tend to forget how they work.

The assumption was that the communications structure would come first and then the design would imitate it, but with really ossified hard-to-change systems it seems plausible that the departmental organization would shape around it.

How often is that the case these days, though? Currently, when I subscribe to a service or buy a product tied to a service, I feel there's a 50/50 chance the company will shut down the service (or itself) while I'm still using it.

    Everybody has a plan until they get punched in the mouth…
    
    - Mike Tyson

Essentially in FP you design your entire program as a single immutable function expression that's made out of a pyramid of layers that's constructed of smaller closed lego like functions composed together. People in the FP world call it the point free style.

The type signatures guide the architecture of your program and can easily be replaced decomposed or recomposed into higher or lower forms of abstraction without compromising your overall program. Design comes naturally like building something out of legos... Easily configurable, and minimal planning or need to hold the entire program in your head. If the design is wrong like legos your program is easily reconfigurable.

The main factor that allows lego like design is immutability. A lego that mutates state outside of its own scope is a lego piece that cannot be modular. FP by making everything immutable makes your program legos all the way down meaning your smallest primitive can likely be broken down further to reuse in other places and recomposed to form other higher order abstractions that are easier to reason about. In both OOP and procedural programming your designs can never have this level of flexibility due in to modules being tied together by shared mutable state.

Of course there are costs to immutability: both low level costs such as memory management and high level costs such as graph algorithms that require mutability to be effective. There are ways to deal with these issues but they are not trivial. In the end all FP programs must "cheat" in some way and have a small portion of their program actually mutate something.

Overall though, in all my years of programming I have found FP to be the best overall answer to the design problems that most programmers run into. It's by no means a perfect answer but it is the best I've seen and unfortunately not applicable to all domains.

FP offers a better syntax for most of the programs I write, hence it makes the initial effort easier. Especially "compiler like" or "interpreter like" programs more often than not just fall into a pit of success.

But still - my original point remains. I am unable to design a program beforehand. The iteration is just faster with FP.

On one point I disagree: " In both OOP and procedural programming your designs can never have this level of flexibility due in to modules being tied together by shared mutable state."

There is nothing that stops one from writing most OO or procedural programs using immutable style. In fact, this style makes those programs better as well. Just try it. Never mutate an object. Each method, if they must mutate something, then let them return a new object instead.

This style is not applicable everywhere, and due to lack of garbage collected immutable datastructures in the C++ basic library, for example, the area where immutable style can be used is smaller than in a FP language.

And you can't use it in those cases naturally where you actually need to mutate existing storage. But, say, an array of 10 elements? Immutable! Most strings smaller than n * 1000 chars? Immutable! Etc.

>On one point I disagree: " In both OOP and procedural programming your designs can never have this level of flexibility due in to modules being tied together by shared mutable state."

When I define OOP I define it as mutable state. The main reason is because there's a equivalence when you do "immutable OOP"

  object.verb(parameter)

  verb(object, parameter)

The difference is (mostly) syntactic sugar and it won't change the structure of your programs overall.

That being said:

  object.setter()

In fact if you do regular procedural C-style programming with immutable variables your code also becomes isomorphic to FP as assignment becomes equivalent to just creating macros for substitution in an otherwise functional expression.

immutable procedural

  def add(x, y):
     doublex = x * 2
     doubley = y * 2
     return doubley + doublex

  def add(x, y):
     return y * 2 + x * 2

  add x y = let doublex = x * 2,
                doubley = y * 2
            in doubley + doublex

  add x y = (y * 2) + (x * 2)

It's all semantics either way and most people do a bit of a hybrid style of programming without ever thinking about what are the real differences so it's not too important.

The thing that's sort of bad about OOP is that it sort of promotes a style of programming where subroutines modify external state that's scoped. You tend to get several subroutines that modify some member variable and this is what prevents you from ever reusing those methods outside of that shared state. <= This is in fact the primary area of "bad design" that most people encounter when doing OOP programming... shared state glues all the lego bricks together making your code inflexible to counter the inevitable changes and flaws that you can never anticipate. The other issue is people never realize that this is what's wrong with their program. They blame their initial design as incorrect when really it analogous to saying that their initial lego project was incorrect and they should have glued the bricks together a different way. The primary key to fixing this design problem is to NOT glue your bricks together at all!

While even though you're capable of the above antics in procedural programming... in procedural programming most programmers tend to keep the mutations scoped to within the boundaries of the function definition hence keeping the function reusable. The downside of this is that if you have shared state within the scope of your function it becomes harder to decompose your function into smaller functions because shared state glues all your internal instructions together.

It's easy to see this face in functional reduce. Higher order functions like reduce allow you to break out the function that actually reduces a list while a for loop with a mutating accumulator can not be broken into further smaller functions.

compare the two below:

loop decomposed into two reusable functions (reduce and add):

  add = (acc, x) => x + acc
  m = reduce(add, [1,2,3])

  m => 6

  m = [1,2,3]
  acc = 0
  for(int i = 0; i < m.length; i++){
     acc += m[i]
  }

  acc => 6

"if you're doing OOP your code will have methods and "setters" that mutate or set internal state"

I don't think there is nothing in OOP that "forces" you to explicitly mutate state. People just happen to do it that way, even if there was no need for that.

I.e. instead of mutating an instance of class Foo

void Foo.set(Some bar)

There are several patterns that make instance of Foo immutable. As an example:

1. Use just 'Factory' pattern to build state and disregard mutation altogether FooFactory fact; fact.AddBar(Bar bar); Foo foo = fact.build();

This may appear as mutating (the factory) but the point is the mutations are located at the factory, which is expceted to have a limited scope, and the entity with larger scope (Foo) is immutable wihtout setters.

2. If there is need to modify existing state, instead of mutating the instance, return a new instance with the value modified. So instead of

void foo.Set(Bar bar)

Use method that creates a copy of foo, modifies state, and then returns a new instance of Foo

Foo foo.Modify(Bar bar)

Just removing mutability actually goes long way in making programs more legible akin to functional programs.

I agree functional style is often the best. Unfortunately we just don't have a functional language that could replace C++.

I never said it forces you to do this. Please read my post carefully.

Let me repeat what I wrote. I'm saying that FP and OOP have blurry definitions that people have an intuition about but have not formally defined. Immutable OOP is can be called FP and vice versa. What is the point of having two names for the same style of programming? There is no point that's why you need to focus in on the actual differences. What can you do in OOP that absolutely makes it pure OOP that you cannot call it FP?

That differentiator is setters and mutators. If you use setters and mutators you are doing OOP exclusively and you are NOT doing FP. My response to your post is mostly talking about this semantic issue and why you need to include mutation in your definition of OOP. Otherwise if you don't than I can say all your descriptions of patterns to me are basically FP. You're following the principles of FP and disguising it as OOP and confusion goes in circles.

Please reread my post, I am aware of OOP and it's patterns so there's no need to explain the Factory pattern to me. I'm also in agreement with you like I said.

I am talking about something different: semantics and proper definitions, you likely just glossed over what I wrote, that's why your response is so off topic.

What about Lens.Setter? (http://hackage.haskell.org/package/lens-4.19.2/docs/Control-... )

On the other hand, the current internet feels like a good approximation of the best trade offs for performance vs. usability in most areas, and I doubt a full rewrite would fundamentally change much about how we use the internet.

An Internet where “keep engagement high so we can serve shitloads of ads” was not the default path to financial success would be an Internet without social media that happily pushes people into ever-more controversial positions, because that’s what keeps them around longer and lets you serve more ads. Decide for yourself what social changes that would create.

Ads still all over.

In contrast to op assuming built in payment would have had replaced ads.

If someone goes full-on "modern internet experience" without ad blockers, third party script blockers, cookie blocking, canvas fingerprinting blocking, etc. etc. the dataset collectable from the user behavior is rather large. And this allows for a detailed profile of the user to be built over time.

But what for? Why so much detail? Why so pervasive collection of online behavior and real-life interests? To sell more things to people? It seems like advertising would work just fine with a much coarser scale. Is it something more exotic, like building a "digital twin" of every citizen out there to predict what they will do and how they behave?

Or can it just be the mundane ads?

Anyway, I've wondered those questions and namely why the profiling part must be so pervasive and massive in scale. Can it not work in lower resolution?

PS. The TV-series Colony contains one possible "answer" but although it is an entertaining and paranoid thought, the real-life plausibility of that answer is rather low :)

As bad as ads and tracking are, they truly enable everyone to use the internet, no matter the background, financial or legal status, age or nationality. In my view, that makes up for all the problems tracking causes.

In areas like music and video streaming, paid services are actually popular, because they have a different cost structure and more consumers find it worthwhile to pay to access better content with no ads.

The problem is indeed greed, but I'm not sure the banks are the ones who are greedy here.

Sure there is, they'd lose enormous money that way. Because the small segment of users that would pay for the average page view are a large portion of the users most valuable to advertisers.

If they charged high enough to make up for the loss of the demographic willing to pay the price, it would be an even smaller group paying very high prices. And even higher prices once you charged extra to recover the costs of developing and maintaining the separate payment and customer support system and service for that small customer base.

Lots of people that are in demographics more valuable than average see stats about the average a site makes per view (or per MAU, or whatever) and says “I'd be willing to pay more than that.”

What they don't realize is that they are worth more than that as advertising targets, and the people that bring the average down to what it is aren't willing to pay the average.

That was extremely low friction, but would still be too bothersome for a micropayment. If any such model wants to have success, it will need to be completely frictionless, with absolutely no interaction required. And it would need to work automatically everywhere, e.g. built into the ad services themselves. There are attempts (e.g. https://contributor.google.com/v/beta) but that's not quite as easy as it should be.

What would happen if I had Apple Pay set to Face ID for confirm?

Is this really an Apple Pay request, or is it a look-alike?

Thanks!!

Perhaps there could be a new digital currency for all and any transactions done online. Let's call it ICR. Websites and apps would only accept ICR, and pay out only in ICR, and people would convert between ICR and whatever they want, and that would be the only way to generate ICR, unlike cryptocurrencies and whatnot.

I imagine that it would in fact have to support people using the internet exactly the way they do now, if it hoped to get traction.

It is far from obvious, TCP has it's problems over fast and relatively reliable networks. For a random example see [1]

[1] https://blog.ipspace.net/2019/06/do-packet-drops-matter-for-...

Sounds like a ML model that can do correlation but can't handle causal and counterfactual reasoning.

Isn't the iPhone/iPad supposed to be exactly that?

A single device, a single ecosystem, a single curator.. and outside of a vocal minority, millions of people love it for exactly those reasons.

It's just a skin on top of the same old stuff everyone else uses, but with more control so you can't reach underneath to see much of the foundation.

That said, it's a fairly deep skin by this point, and getting deeper, but compared to the whole depth of the stack we're talking about, it's still just the skin.

While I don't know enough to comment authoritatively on it, I think with respect to this Apple might have taken a step backwards (at least with respect to percentage of system that is unique, if not in total amount of libraries and code) when it switched to OS X. OS 9 and its prior versions were an entirely unique and custom OS designed at Apple. The whole switch to OS X was taking a UNIX core for stability and building on top of it.

Apple systems are quite different than the norm in many respects, but the scope of this discussion is so large that there's really little they do that's different than everyone else from that perspective, because nobody does anything different. It's hard to start from scratch and ignore approximately 80 years of concerted computer science effort and come up with something useful and usable in any sort of time frame that makes it worthwhile.

I’m a software engineer, I’m perfectly well aware iOS and MacOS run on top of a Unix kernel. But unless I go looking for it, they will never shove this in my face and make it my problem.

Meanwhile, I’m still dealing with Linux-loving colleagues who cannot get their sound or webcam to work reliably and consider this an acceptable state of affairs.

That's largely the fault of manufacturers who refuse to properly document their devices and release functional drivers. I don't think anyone considers that an acceptable state of affairs, merely tolerable if it allows using an open software stack that you can freely modify.

This isn't so much of an issue anymore in my experience. Older webcams and sound cards that work flawlessly with Linux often don't work with newer Windows versions, though.