edsiper2

Calyptia | https://calyptia.com | Full-time / Remote (Americas / Europe) preferred

Calyptia is the company behind Open Source project Fluent Bit[0], with a strong DNA in Open Source, we solve Observability challenges around scalability and processing in data pipelines.

We have couple of open roles:

- Engineering

  - Senior Support Engineer (Linux, Kubernetes)

  - Senior DevOps Engineer (Cloud, Kubernetes, CI/CD, Automation, etc)

  - Senior Software Engineer (Golang)

  - Senior Software Engineer (C / Linux - userspace)

- Marketing

  - ABM Demand Generation Manager

- Sales

  - Sales Development Representative (SDR)

  - Senior Sales Engineer

- Others

  - if there is no open role for you but you think you can navigate with us and take Calyptia to the next level, send me an email: eduardo at calyptia dot com (and..tell me why ?)

To apply to any of the listed jobs, please do it through the following Careers page, YOU MUST ADD THE FOLLOWING HASHTAG in the "Additional Information" section: #iamtheonefromhackernews:

- https://calyptia.com/careers/

.

[0] https://fluentbit.io

Naming is hard, but please, do not repeat the mistake of many OSS project in the last 20 years calling each project by prefixing the name with the stack/environment involved.

Now a "trending" language can catch the attention, but tomorrow?.. maybe. So the value proposition and starting from it name should be different (if you want adoption).

For my use case, for a rewrite of jq I would expect one thing only: higher performance... show the numbers ;)

Calyptia Inc.| Senior Frontend Engineer - Lead| REMOTE | Full Time

Calyptia is an open-source company that focuses on solving Observability challenges for cloud environments. We are the creators and maintainers of Open Source projects Fluentd and Fluent Bit, which are all about observability (logs, metrics & traces).

We are looking for an experienced Frontend developer that has been a tech-lead before with strong skills in architecture design, TypeScript, React, GraphQL, Git and API services,

We look for a proactive candidate with strong communication skills and able to work in a distributed team.

If you are interested, drop me a message to eduardo at calyptia dot com

First of all congratulations to Valgrind and the team behind it! This is an essential tool that help me personally over the years while developing.

What needs to be done to get Valgrind binaries available for MacOS (M1) ?, from a company perspective we are happy to support this work. If you know who's interest and can accomplish this pls drop me an email to eduardo at calyptia dot com.

ah!

[ERROR ] GitHub rate limit exceeded! If you have not done so already,

[ERROR ] you can tell kraft to use a personal access token when contacting

[ERROR ] the GitHub API. First, visit:

Calyptia | REMOTE | Full-time | https://calyptia.com

Calyptia is the company primary funding the OSS projects Fluent Bit[0] and Fluentd[1], CNCF graduated projects.

We are looking for talented individuals with outstanding discipline to work on solving challenges associated with data processing and scalability. We are looking for candidates for the following teams:

- Backend & Ops: Golang, Kubernetes, CI/CD

- Frontend: Javascript, Typescript (mandatory), React

- Fluent Bit OSS: C, Linux and familiar with networking, event loop, syscalls, protocols, etc.

Feel free to send me your github profile or resume to eduardo at calyptia dot com.

[0] https://fluentbit.io

[1] https://www.fluentd.org

my 5c about your intro: I see you can handle a bunch of technologies but your primary focus and interest in the last time has been on Rust. You might promote yourself as a Rust Developer instead of mentioning all the other tech, so you can get easily filtered for the right job.

msgpack users:

- Fluentd & Fluent Bit

- Microsoft SignalR https://docs.microsoft.com/en-us/aspnet/core/signalr/message...

the following as a 5 min intro video to one of the techniques :

- https://www.youtube.com/watch?v=FP-dNd1xTcs

if you want to learn more: https://yogananda.org/practice

Lua is just great. In Fluent Bit[0] we expose Lua filter capabilities. So end-users can write their own scripts to manipulate record logs and accommodate keys/values easily with a simple programming logic:

- https://docs.fluentbit.io/manual/pipeline/filters/lua

I am happy to see performance is being improved, as of now we stick to LuaJIT, since our use case is pretty simple we have not found a good reason to move away from it.

[0] https://fluentbit.io

Logstash alternatives with Elasticsearch support:

[1] https://www.fluentd.org (Ruby Language)

[2] https://fluentbit.io (C Language)

Log structure is really important, from the examples provided I would suggest the same approach can be used using a full 'logfmt' style, so timestamp and the event type can be set as keys, e.g:

  ts="2019-03-18 22:48:32.990" event="Request started" http_method=POST http_path=/v1/charges request_id=req_123

the main difference is that you make easier the parsing since many tools can parse lOgfmt without problems.

One interesting use-case here for 'me' is the ability to perform queries in a schema-less fashion and I will do a quick speech on what we are working on Fluent Bit[0] (open source log project), pretty much the ability to query your data while is still in motion (stream processing on the edge[1]). Consider the following data samples in a log file:

  ts="2019-03-18 22:48:32.990" event="Request started" http_method=POST http_path=/v1/charges request_id=req_123
  ts="2019-03-18 22:48:32.991" event="User authenticated" auth_type=api_key key_id=mk_123 user_id=usr_123
  ts="2019-03-18 22:48:32.992" event="Rate limiting ran" rate_allowed=true rate_quota=100 rate_remaining=99
  ts="2019-03-18 22:48:32.998" event="Charge created" charge_id=ch_123 permissions_used=account_write team=acquiring
  ts="2019-03-18 22:48:32.999" event="Request finished" alloc_count=9123 database_queries=34 duration=0.009 http_status=200

so if I wanted to retrieve all events associated for user 123 I would process the file as follows:

  $ fluent-bit -R conf/parsers.conf \
               -i tail -p alias=data -p path=canonical.log -p parser=logfmt \
               -T "SELECT * FROM STREAM:data WHERE user_id='usr_123';" 
               -o null -f 1

the output is:

  [1552949312.991000, {"event"=>"User authenticated", "auth_type"=>"api_key", "key_id"=>"mk_123", "user_id"=>"usr_123"}]

the results are in a raw mode but can be exported to stdout in json, to elasticsearch, kafka or any output destination supported.

One of the great things of the stream processor engine is that you can create new streams of data based on results, use windows of time (tumbling) for aggregation queries and such.

[0] https://fluentbit.io

[1] https://docs.fluentbit.io/stream-processing

Yes

A really good library that we use is libco[0], it supports different architectures and mechanisms. We use it extensively on Fluent Bit[1] to manage async IO network operations (epoll + coroutines).

- [0] https://byuu.org/library/libco/

- [1] https://fluentbit.io

Since your goal is to have a career switch I would recommend you the following approach:

- Essential books and papers are good as a reference to learn and improve but focus first on hands-on stuff.

- Get involved in an open source project that fits the 'system level' category, learn how it works and try to contribute to it (again, put your hands-on)

- Start with development in C language, a good tool to build would be a TCP client and server (kind of 'echo' program). The goal of this will be to learn: how to compile a C program, basics of memory management (malloc, free, realloc), networking (create a socket, bind, listen, connect), transfer data (write, read) and so on.

- If possible, learn using a native Linux environment.

All of the suggestions above will pay off if you are persistent and keep trying learning by building; of course, getting involved in an open source project is a must if you want to have a successful career.

Getting into open source very early

in my experience I would consider the following:

- Understand what are your takeaways for the audience. Remember it's about the content, keep in mind main areas/topics to cover.

- who's your audience ?, there are a few conferences where going very technical (low level) matters, most of them want to focus on how X works, how to solve Y, etc.

- "Start with why". Yeah, it should sound familiar, check this 5 minutes video: https://www.youtube.com/watch?v=IPYeCltXpxw .

- Note that likely in the first 3-5 minutes the audience will have a > 90% focus on what are you talking about, make sure to take advantage of that time with a good "why" (see above) for engagement.

- Share your presentation slides with a colleague and wait for "honest" feedback.

we are using libco[1] in our project[2]; it provides a good abstraction for architectures and provide different backends using ucontext, setjmp and native register management for x86 and x86_64.

[1] https://byuu.org/library/libco/

[2] https://fluentbit.io

check the GIT commit changes listed in the post, you will see the details there, e.g:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

Definitely:

5. The odd Redis persistence model: fork() and fighting the copy-on-write.

FYI: The tutorial guide is missing: http://lcui.org/guide/

is there any big difference with the Debian image provided by Google ?

   launcher.gcr.io/google/debian8

(besides debian version)