The backend had to receive events from a webhook and accumulate statistics on those events, then serve these statistics via REST API
Frontend was plain old React reaching the REST API.
Issues:
* The database (MS SQL Server, as recommended by Microsoft) would randomly lock up for days. Bear in mind this was a 1GB of RAM (the minimum amount that Microsoft accepts as enough to run a SQL Server instance) RDS serving around 600 MB of data.
* The introduction of the Repository Pattern (as recommended by the ASP.net documentation) introduced a lot of complexity (including incompleted/non-defined operations) that slowed down significantly the development.
* LINQ supports interleaving DB operations with not-db operations, quickly becoming a O(n^m) footguns. If you make `row => row.x > 5` a method, LINQ will happily download all the rows and run the comparison in program instead of in the DB. Bear in mind I can't construct these footguns with django ORM, or make them typecheck with any Haskell data access framework.
* The test runner UI would pretend no tests existed if the code didn't compile for any reason. Trying to debug something by running a specific failing test? If you miss a semicolon now you have to go find your test again.
* The logger would log the message and the affected class in different lines. Do you want to grep for an error? You better know the obscure grep syntax for "previous line". Want to change the logging format? Good luck finding the code that adds the newline in order to find how to influence it. If you ask the IDE to find the implementation of the logging function it will show you the interface, nothing more.
* You better not do some type of async stuff in LINQ, because the system is going to spawn a process for each entry, obliterating your production instance by creating thousands of processes, using a bunch of MB for each process.
There were more issues, I didn't make an exhaustive list of them because, you know, I don't need to make exhaustive lists of malfunctions and bad design ideas for other programming languages.
Microsoft doesn't force you to use SQL Server. I used Postgres and Redis for a quite big microservice based app.
Repository isn't mandatory, however is nice to separate the business logic from the details of the data store or database. Using a repository shouldn't be a a problem unless you are doing it wrong. I've seen enough of poor repository pattern implementation. I would question the skills of the developers here.
LINQ was not used properly if you had mixed IEnumerable and IQueryable based queries. It's not frameworks fault here, I'm afraid.
The system is not spawning threads for async requests, it's just creating Tasks which cost very little since they aren't physical threads. Think of goroutines, not threads.
All the things you mention seem to be due to lack of experience. I am not sure how using another platform would have yield better results, skill sets remaining the same.
It's not a problem if you try a new language, platform or framework as long as you don't just churn out code assembling method calls based on name (. NET makes this very easy because of intellisense) and examples found on Stack Overflow. As long as you understand what the code does, you are fine.
I think I'm going to go with frameworks that have the actual best reccomendations baked in, instead of the frameworks that recommend footguns.
For every good framework, regardless of platform and language, there is going to be a learning curve. That's because flexibility and extensibility add complexity.
Only a trivial framework with limited use cases would not allow you to make mistakes. I take it you never used C or C+=, because there's a footgun waiting for you at every keystroke. :)
https://docs.microsoft.com/en-us/sql/sql-server/install/hard...
Am I unwise for following the official Microsoft documentation for a Microsoft product?
Yes, I'm unwise for following it. I shouldn't have accepted the project because it required Microsoft technology. That would be wise.
Also, just sounds like you hate Microsoft, and are looking for (poor) excuses to continue your hatred.
Was the server running Windows? Windows doesn't perform really well in 1G of RAM, especially if you're running GUI Windows and not Windows Server Core. Then running SQL server on top of that, well that 1G RAM probably IS the performance issue. A good rule of thumb is Window takes 3G, and your programs go on top of that.
There were clearly other issues from what the OP described though, and to use LINQ you do have to have a good clear mental model of the runtime actions that it give you. It's a power tool, and can let someone experienced develop very fast, or let someone inexperienced cut their fingers off equally fast!
To provide an answer to OP, about following MS recommendations .. the reality is that not all MS advice is good advice, and also that with so many versions not all of their documentation is up to date. However it seems your team did draw the short straw! Mostly their documentation is very good for product and API level documentation, not as good for architectural documentation.
In the .Net world, if MS released a certain library, that would become the de facto choice for nearly everyone in the ecosystem. Competing with the MS options was almost impossible.
So even if the MS choice had a lot of flaws, it would remain predominant.
Which meant that as long as the MS option was good enough, you wouldn’t see too many alternatives unless they were an order of magnitude better.
And a large reason for this was the closed source nature of .NET.
In the open source world, even if the primary vendor option is great, but it has only a single minor issue, someone will invariably fork it and fix the minor issue. Which puts competitive pressure on the primary vendor to incorporate that fox, or if they’re unable to, see a lot of people migrate to the competing, slightly better offering, which is easy to do since it’s basically the same product with that 1 slightly better fix.
In practice the original vendor almost certainly incorporates the Fox, especially since it’s so easy to do as someone else has already written the code for it.
So IMO the problem with the .NET/C# ecosystem used to be it’s closed source nature, which led to very little competitive pressure for improving tools and libraries, unlike all the primarily open source alternatives you mention.
While I get your point, I personally prefer it this way. Whatever Microsoft will put out will at least be “ok”, regularly bug fixed and supported for ever and ever.
I prefer to having an ok default and just rolling with it.
But this is a tacit agreement in the .NET world. It's not only about libraries written by Microsoft. When sensible libraries such as Automapper or Newtonsoft JSON appear, everybody defaults to them.
A big thing that people often don't understand about .NET is it has the perspective of being all things to all people.
One example of this is that it gives first class support for supporting living extensible systems for long durations like decades which is often not really the case for other stacks. Hence the prevalence for approved long term supported libraries typically with first party backing from MS.
Another aspect of this all things to all people is that it has lower level closer to the metal performance features that people only used to high level stacks often mention they don't see the need for (although they don't say it like that!).
Another aspect is you can do pretty much any kind of application in it, from web to native to high performance games or latency sensitive systems (with some special techniques) to well almost anything. The ability to access low level performance while still having high level general ergonomic APIs is critical to this.
Footguns are common.
I am not sure where the .NET community coalesces (python and similar tend to hit up IRC).
I don't understand the dotnet's extension ecosystem -- how to find quality sourcing, etc. Going from an established project into dotnet SDK makes sense. But searching GitHub for useful dotnet frameworks -- stars are probably misleading here?
I’m sure there are issues with learning curve but the ecosystem is very capable (the corollary to being very productive) but you need to know how to properly use it.
My ego is not unfounded, I have had to do this successfully in other projects, but I can only do it if both the tools, the codebase and the framework allow me to do it
Your explanations of what went wrong show clearly that you still don't have a grasp of how async/await works. So you don't understand the concept, which is now common across a number of languages including javascript (which you claim to know) and python (which you claim to know), dart, and scala.
Based on your technical knowledge of languages you claim to know, I would say you are closer to a junior developer.
If I'm a junior, 99% of the professional programmers out there are simply not programmers at all.
I feel you on the hiring bit, there's such a glut of resume padders, bullhockey purveyors, enterprise leftovers in .Net it's hard to find good people (especially in the US). If you put feelers out to any recruiting firm you'll receive a who's who of people no one will hire. There's also not much of a community to speak of compared to other platforms (what community there is is spread out across numerous platforms or centered around MS, github) so there's not really anywhere you can post your jobs and expect a positive result.
To anyone trying to hire heavy lifting .NET expertise, I am now looking for an interesting fully remote role!
I've been using .NET since V1 and Delphi before that so I have deep exposure to getting things done with this stack and would prefer to work with it compared to most others. I've also worked commercially with about 10 languages so fairly polyglot, just find C# very powerful with a focus on GTD.
> LINQ supports interleaving DB operations with not-db operations
This hasn't been the case since EF Core 3.0, it will throw an exception if something can't be translated to prevent this problem. Prior to this (EF Core 2.2), a warning would be spat out if an in-memory operation occurs. Due to this, I assume this project was a number of years ago.
> The logger would log the message and the affected class in different lines.
The default logging format is unfortunate, although it can be made a single line in .NET 5 onwards. This issue would've been easily solved by using Serilog, swapping out the built-in logging with a few lines of code. I don't know the last time I encountered a .NET dev that wasn't at least aware of Serilog as an alternative.
So it looks great for a blog post, for a linkedin thought leader article, but when actually used, you end up duplicating the same shit over and over and over unless you use extension methods. But then why having the damn Repo patterns with the basic methods in the first place?
These patterns are just awful and don't solve anything, abstraction of the DB? You don't need that pattern for that.
It's true, EF abstracts the database away by not requiring you to write SQL but EF is only usable with some SQL databases.
So, having another abstraction layer might not be a bad idea.
You can't. There is no such thing as switching databases "with ease" for any serious application.
Postgres, Redis and Cassandra are all very different databases and it doesn't make sense to switch between them; unless your app is small and still experimental so the changes are trivial, or so large that it's a major project with very specific needs that requires far more work than a repository would solve.
But if you really want to, Entity Framework already has multiple providers including non-relational databases like Cassandra. Anyone can write a provider and there are many 3rd-party libraries for whatever data source you want.
> "having another abstraction layer might not be a bad idea."
It's a terrible idea. It's such an exceedingly rare situation that you're wasting a lot of upfront work for something that'll never be used.
Also .NET has a fantastic data system with LINQ which is built around `IQueryable` and provides powerful query modeling that can be incrementally and dynamically changed before its pushed down to the datasource. It's what makes EF so good and why repository patterns cause so many problems by breaking this interface for no productive reason. You can also use extension methods on the `DbContext` to extract complex queries or large operations to a single method.
Now all that being said, there are times where you can have an abstraction/interface to your data sources if they're very complex or spread across multiple databases or have other processing needs. But at that point you actually need it, instead of just doing it as a "pattern".
Then in startup.cs or program.cs, add this to your services: optionsBuilder.UseCassandra("Contact Points=127.0.0.1;", opt => { opt.MigrationsHistoryTable(HistoryRepository.DefaultTableName, "<schema name>"); });
See more here: https://efcorecassandra.readthedocs.io/en/latest/intro/getti...
* SQL Server can be incredibly fast with good access patterns, but the rest suggests you didn’t have those. * Yeah, the repository pattern is garbage. I prefer just define the operations you need and use Dapper. * I know some on here are wild about Entity Framework, but personally I’d rather write the SQL myself. You need a good mental model of what’s going on under the hood or what you’ve described happens. * Honestly never used the default test runner because it seems pretty bad. NCrunch, however, is incredible and doesn’t suffer from this. * I don’t think I’ve ever worked anywhere that used the default logging implementation and yes, that format is weird. (You can, however, step into the logging code if you want.) * Depends on what you’re doing, but yeah 10,000,000 Task.Runs is not going to perform well. But I’m not sure the equivalent code works well anywhere except possibly BEAM.
So whilst none of these things cause me problems on a day-to-day basis, I can 100% see how inexperienced devs could drive straight into that tar pit.
The old C# stuff is indeed crusty, intermixed with hot nonsense from the Balmer days.
Latest Framework version is I'd say OK, nothing to write home about, newer stuff is much better (.net 5, 6).
I do think MS SQL Server is a bit, well, bad. But don't worry, irrelevant to the C# stuff - I used to use java servlets to query mssql, while it was better performing than mysql, I wouldn't pick it again.
The anecdote about the React frontend bit is interesting, but ultimately depends on what the UI was doing. JS is certainly good to bang out some quick UI, but if it was shell for the backend, I'd say your app was either mis-designed or just naturally backend heavy.
Again, haven't used (not in web dev right now), but Blazor exists for doing C# web UI, instead of complaining that the backend was bad/hard a better comparison would be between Blazor and React.
Or you could switch architectures and do a non-component-based web UI - faster to put together a mashup of JS junk but it doesn't necessarily perform as well as doing a web client old-style - simple request response works for a lot of use-cases, its not as flashy, but it is fast.
This is no longer the case since EF Core 3.0. Only top level projection can contain expressions that aren’t translatable to SQL.
> You better not do some type of async stuff in LINQ, because the system is going to spawn a process for each entry…
I wonder exactly what you ran into. This is not an accurate description of how EF Core’s async support works. Not even if you replace the word “process” with the word “thread”.
What were the actual issues and what would've made it more productive? That's much more useful for discussion here.