News for hackers. Stuff that matters.
Re:Introducing Vector: Netflix's On-Host Performance Monitor... by CWuestefeld on Wednesday April 08, 2015 @02:01PM (#9342610)
It's enormously frustrating to me that the Windows platform is so far behind.

I've been researching for a rebuild of our ecommerce site with the idea of a modern microservice architecture. Team skillset and some other considerations dictate a .Net environment.

Netflix and other companies have created a really rich platform, between logging and monitoring technologies like this; message queueing; deployment; and so on. In the Windows environment there's precious little to compare to.

One bright spot is the news story also on HN today [1] about MS announcing Docker-related Hyper-V technology - but in the next version of Windows server.

It might be that to satisfy those .Net compatibility wishes, we just go to Mono and do everything else in Linux.

[1] https://news.ycombinator.com/item?id=9342369

Re:Introducing Vector: Netflix's On-Host Performance Monitor... by AloisMayr on Thursday April 09, 2015 @07:46AM (#9346870)
Want to try https://ruxit.com for monitoring your .NET ecommerce shop on Windows? It also monitors your host and real user interactions with your site.
Re:Introducing Vector: Netflix's On-Host Performance Monitor... by dkhenry on Wednesday April 08, 2015 @08:44PM (#9345144)
So I haven't looked into all the metrics that they are getting off of PCP but from an initial glance it doesn't look like they are getting a lot more information then you get off net-snmp.

In this one case the windows platform isn't going to hold you back too much. On top of that anything the .net stack on windows gives you access to WMI which can give you really simple really deep data from your application.

Don't buy into too much hype on what some of the big guys are doing on monitoring, most of their challenges seem to be around the scale they are monitoring at not with the depth of metrics.

Re:Introducing Vector: Netflix's On-Host Performance Monitor... by brendangregg on Wednesday April 08, 2015 @09:41PM (#9345348)
For the initial release, you are right that you'll find most of these metrics in net-snmp. It's the selection that's more interesting, which is guided by the USE method: identifying utilization, saturation, and error metrics for key resources. We're missing some metrics (so do all monitoring tools), and we'll be extending Vector to include them (as can other people, as it's open source).

One challenge is that there are some metrics that the kernel doesn't make available, so we need to either use tracers to fetch them, patch the kernel, or use custom kernel modules. In recent weeks, I've been using (or trying to use): ftrace, perf_events, SystemTap, eBPF, and LTTng. I've also used others, but not in the last few weeks. I'm really looking forward to getting these advanced tools into Vector.

Re:Introducing Vector: Netflix's On-Host Performance Monitor... by tonyarkles on Wednesday April 08, 2015 @08:06PM (#9345013)
> Team skillset and some other considerations dictate a .Net environment.

Not to be a dick but... that doesn't sound like a Windows limitation.

Re:Introducing Vector: Netflix's On-Host Performance Monitor... by voltagex_ on Wednesday April 08, 2015 @07:32PM (#9344885)
I suspect that you could extend PCP (the monitoring component of this dashboard) to support Windows' performance counters. The profiling capabilities are certainly there.
Re:Introducing Vector: Netflix's On-Host Performance Monitor... by nathan_scott on Thursday April 09, 2015 @10:01PM (#9352185)
PCP can export many hundreds of Windows-specific counters (which it accesses via the PDH APIs). The Windows build does need some love though, as the OP mentioned - its out of date and needs a regular contributor to bring it up to date with the latest PCP sources.

Then Vector should Just Work (tm) for Windows servers too.

Re:Introducing Vector: Netflix's On-Host Performance Monitor... by yareally on Wednesday April 08, 2015 @09:07PM (#9345218)
> Netflix and other companies have created a really rich platform, between logging and monitoring technologies like this; message queueing; deployment; and so on. In the Windows environment there's precious little to compare to.

Have you looked at any of the following for the various gripes you mentioned? I came from a primarily Linux background until recently, but I know .net and Windows have good tooling that extends beyond my modest list below. Looking to Microsoft to provide every option is probably not the best way to go about it, not sure if that was mostly what you were looking at so far.

https://www.rabbitmq.com/install-windows.html (company I work for uses it for messaging between their services)

http://nlog-project.org/ (we also use that as a base framework for our logging)

http://getakka.net/ (never used it on .net but can't do without it in the jvm)

https://github.com/dotnet/Orleans (sort of like Akka, but with additional features)

https://github.com/MSOpenTech/Redis (it's a bit behind the nix version, but still well maintained)

https://msdn.microsoft.com/en-us/data/gg577609.aspx (Netflix uses the Java port of that, fyi)

https://technet.microsoft.com/en-us/sysinternals/bb896653.as...

http://en.wikipedia.org/wiki/Resource_Monitor

https://www.jetbrains.com/dotmemory/

https://www.jetbrains.com/profiler/

http://www.appveyor.com/ (great for CI and easy to set up)

https://www.cygwin.com/ (when you miss a nix shell and tools)

http://en.wikipedia.org/wiki/Windows_PowerShell (for everything cygwin can't do easily)

Re:Introducing Vector: Netflix's On-Host Performance Monitor... by wantab on Thursday April 09, 2015 @07:00AM (#9346722)
The web was built, and runs on, *nix/BSD. Windows is an outlier. Windows can't even get the slashes going the right way. There's a reason 80% of internet traffic does not use Windows.
Re:Introducing Vector: Netflix's On-Host Performance Monitor... by CWuestefeld on Thursday April 09, 2015 @10:26AM (#9347730)
I'm not sure how that was supposed to be helpful.

I also don't think it's strictly true. For sure the underlying networking came from Unix. But as for the web itself, once we got to real dynamic content, it seems to me that Microsoft were the ones that got things moving.

While the Unix world was mired in the awful world of CGI, Microsoft gave us high-performance ISAPI, and then Cold Fusion (also on the Windows platform) and Microsoft's ASP made programming a little more sane. While the Unix world tried to deal with JSP (which IMHO wasn't a very good solution), the Microsoft platform seems to have been the innovators for several years, until Ruby on Rails and then node.js and stuff started coming out.

Today, the Apache server powers far more sites than any other, it's true. But IIS shares the 2nd-place spot [1]. When you say "There's a reason 80% of internet traffic does not use Windows", that's pretty much true for the servers, but far off the mark when counting clients. And the reason for that is that Microsoft's strength hasn't historically been radical advances, but in figuring out ways to take the bleeding edge tech that doesn't really work quite right yet, and packaging it into commodity software that may not be as sexy as envisioned by those with the original ideas, but actually useful to the average guy.

[1] http://www.zdnet.com/article/web-servers-microsoft-iis-and-n...

EDIT - missing word "world" in 3rd para

Re:Introducing Vector: Netflix's On-Host Performance Monitor... by wantab on Friday April 10, 2015 @04:29PM (#9356928)
Your examples presume there was something better than CGI at the time and the other products are better than something else. For one, I wouldn't be caught dead using any of the products you mentioned.

You claim IIS by using a ZDNet article from two years ago but the reality is IIS is number three behind Nginx and Apache. Still, being a distant #2 is nothing to brag about.

Now you're trying to claim clients are what powers the web but that's not the topic. What an amateur uses does not define what the professional uses. And to claim not wanting to be on the bleeding edge of things is no excuse for falling behind. Firefox and Chrome knocked IE off its perch years ago by being on the bleeding edge.

Re:Introducing Vector: Netflix's On-Host Performance Monitor... by SEJeff on Thursday April 09, 2015 @10:19AM (#9347676)
Likely more than that seeing as how the majority of the load balancers and many of the routers / switches out there run some form of BSD (netscalar) or Linux (Arista), etc.