frontierkodiak
You can keep an eye on (gh) polli-labs/linnaeus (a bit stale; I'll rebase on my private repo later tonight-). There's some cool ideas in here to exploit the structure of taxonomic hierarchies to help the model approach recognition how a professional taxonomist might.. so working from coarse to fine, taxonomy-guided label smoothing (distributing alpha mass by taxonomic distance)..and (forthcoming) RL on expert consensus to teach abstention (if an expert could only identify a specimen to genus for some set of inputs; then our model should abstain from a species classification for the same inputs). Unfortunately I am very, very compute-constrained- but shooting for late April/first week of May for insect + flowering plant models. (Other taxa will come later; probably as unified model). I'm working on camera-based (automated) ecological monitoring systems for ~6yrs at this point; it's a really fun problem space! dropped out of grad school to go all-in on automating my favorite job I ever had (pollination ecology field research..watching flowers for visitations!); since I knew I'd always be a mediocre ecologist- but an engineer that happens to care about ecology could be very very valuable to my field.
a taxa recognition model turns out to be only a small piece of the system you need to extract structured observational data from cameras in the field :-) Working with one of my partners right now to launch a really cool demo of what's possible these days- Texas folks especially; keep an eye out on wildflower.org around May 1!
I'll spill more ink soon but (anyone) please get in touch if you find these things interesting. Or if you'd like to help me out with compute/expenses!
I have a small wrapper around rip2, aliased to `recycle`; files go to a `graveyard` zfs dataset. I deny `rm` usage for agents, a simple (global) instruction pointing to recycle seems to do the trick for Claude.
Seems like a quick win to remove some downside risk and make me a bit more comfortable letting agents run wild in local workspaces.
but as you mention, labeled data is the bottleneck. eventually I'll be able to skirt around this by just capturing more video data myself and learning sound features from the video component, but I have a hard time imagining how I can get the global coverage that I have in visual datasets. I would give anything to trade half of my labeled image data for labeled audio data!
I originally used the Backblaze when selecting the drive I'd build my storage pool around. Every time the updated stats pop up in my inbox, I check out the table and double-check that my drives are in fact the 001Gs.. the drives that Backblaze reports has having 0.99% AFR.. I guess the lesson is that YMMV.
Typically this might mean that you simulate an 8bit forward pass to ensure that the model is robust to quantization ‘noise’. You still use FP16/32 for backward pass & weight updates for numerical stability.
It’s just a way to optimize the model in anticipation of future quantization. The experience of using an 8-bit Nemo quant should more closely mirror that of using the full-fat bf16 model compared to if they hadn’t used QAT.
I may have some more time to debug my trace tonight (i.e. remove conditionals from model + make sure everything is on CPU) and will update if I have any new insights.
``` from torch.export import export ... example_args = (dummy_input) exported_program = export(model, args=example_args) ```
Links: - torch.export docs: https://pytorch.org/docs/stable/export.html#serialization - Using 24.02 container: https://docs.nvidia.com/deeplearning/frameworks/pytorch-rele...
I'm working on something to let you measure animal activity without pasting QRs [1]. I've been running a casual study of my bird feeders since September, and my system will be field-deployed by a few labs this summer. My background is in pollination ecology, so bee/pollinator tracking is a top priority. If you want to study your own backyard, you can use polliOS with your own IP cameras. Targeting March 1 for beta release, but you can submit your email to get notified.
[0] https://github.com/jamescrall/BEEtag [1] https://polli.ai
Some ESP variants have a built-in U.FL connector which makes things easier (e.g. AITHINKER ESP32-Cam and its many clones), but you still have to remove the 0Ω resistor, which I find very difficult with even the smallest tips on my Hakko iron. Could be a skill issue on my end; should be much easier if you have a simple reflow/rework station.
I’m working on a tool to make insect (focused esp. on pollinators) monitoring 10,000x cheaper. We do not currently understand the scale of insect declines, much less are we able to design targeted interventions. In the past decades, mammalian and avian declines were largely reversed, mostly thanks to ambitious multiyear monitoring efforts that enabled limited conservation resources to be allocated efficiently.
We should replicate this model with insects. I plan to make this possible by radically reducing the cost of data.
I’ve been toying with this idea for a few years, but I finally made the leap to building my startup^2 full-time a few months back (coming from entomology/ecology). Hoping for an alpha release & demo study this summer.
I'm happy to chat with anyone interested. Especially interested to meet those working and funding in this space.
1. https://www.nature.com/articles/s41559-020-1269-4 2. https://polli.ai Note: This site is austere (I blame CUDA troubles for stealing my time this month). Expect something more representative by April.
Not sure how well-evidenced this is, but something I've heard from relevant researchers is that bison hooves are more scoop-link vs. cattle hooves. Bison more thoroughly till topsoil and activate microbial/fungal processes that promote floral diversity (among other things).
My present situation is not all that dissimilar to what you describe.
I have been in a MSc program for the past 2 years which has continually threatened to drain all the life-force out of me. I’d always been a happy-go-lucky, jubilant, highly social person, but I moved 1000mi away from my friends and family to accept what appeared to be a funded position which perfectly aligned with my interests. It took me maybe 6 months to realize that I’d fucked up. You might be able to relate to some of the reasons.
For the first time in my life, I was alone, all day, every day. I had zero energy. It took all the effort in the world to get out of bed before 5pm.. I was getting up at 6am to climb mountains every day only a few months before!! I would go weeks at a time without talking to another person, because they simply were nowhere to be found. ‘My’ department had, of course, moved almost entirely online, and they didn’t know I existed. I developed a horrible mental dialogue of constantly shaming myself for my lethargy. I didn’t let myself do my hobbies, since to my mind I hadn’t ‘earned’ them, since I wasn’t getting any work done. I waited it out for a bit, I thought that surely the department would come back to life, and I would finally be able to make human connections, and I would just revert to the old me. Of course, this didn’t work, because nobody wanted to come back: all the decision-makers were happier than ever sitting at home with their families all day. By the end of my first semester, the terrible dullness which has overtaken my person wasn’t going to just go away.
And yeah, I think having difficulty with executive function leads to really harmful positive feedback loops when you’re not doing well. As I continued to fail to get work done, and my mind became less and less healthy, my excitement and joy for the world and for my work was replaced by anxiety and despair. I stopped sleeping well, I stopped eating well, I stopped giving a shit. I spent damn near two years sitting in front of a computer, haranguing myself for not being better, my internal voice screaming at me that if I could just be a professional for this day, and get my fucking work done, then I’ll feel better. It doesn’t really work that way. I’ve also got ADHD, but I’d always done quite well in school etc. because I had good strategies for managing my mind—- I knew that if I surrounded myself with other people, then I’d always been energized enough to get my stuff done, even if it wasn’t at 100% efficiency. I didn’t mind, it worked very well for me— I’d occasionally hand something in late, but I earnestly didn’t give a shit because I loved the way my mind worked: I was always overflowing with energy and creativity etc., more so than my peers, so I figured I was doing something right, and embraced that part of my cognition. All that goes out the window when I moved across the country to start my MSc: all of a sudden, the joy of befriending new folks, of working in a high-energy environment with other folks who are passionate about their work, that stuff all disappeared. The people at my new department didn’t know me and they actively did not want to know me. I was just a threat to them: the casual interactions which had made me love life were eliminated. The only connection that I had to people at my new school were over email and over Zoom. I felt helpless, I knew exactly what was wrong in my life, but there was nothing I could do to fix it. I despaired. I felt like I was dying.
I wasted 1.5 yrs before I finally decided to reclaim some agency in my life. I earnestly weighed quitting. I didn’t need the degree, I went to grad school for ‘everything else’ and all those other things had been denied to me. I would make far more $ if I just went and got a software job. And most importantly, I realized that I owed my advisor nothing: I had wracked myself with guilt that I hadn’t been producing for him, even though he was paying me a stipend.. but I finally valued myself and realized that I needed to place value on what I get out of my early 20s. I plotted realistic escape routes: I could just quit, go home, recharge with my family, furnish up my GitHub etc. and go get a good job. Whoa! I realized I was excited! Excited about a possibility in my life for the first time in ages, rather than just passing time, waiting for someone to give me permission to be happy. This was insanely liberating. I finally had another option; I knew I would be okay if I quit. And so before this spring semester, I basically told my advisor that I’d quit if I couldn’t work remotely for this semester. I told him I had waited too long. I needed more from my life. I understand that they do not want to get COVID, I don’t either, but I need to value my health, too, and my mental health is a huge part of that. He acquiesced, and I worked up some extra scratch so that I could go spend time with friends, family, my girlfriend for the past 6 months. I’ve gotten 10x more done in this span than I did over that stupid year and a half where I was just in a room alone all day, every day. And most importantly, I’ve just got life in me again. I have thoughts and feelings and ideas and feel excitement about the future. I’ve got so many passion projects lined up for when I finish in about a month. My executive dysfunction is still pretty crappy, but it’s worlds better than it was, and I’m going to finish. Just barely— but I will. I’ve also got an ‘F’ incomplete class from a breakdown in a previous semester, which I am completing now while also finishing my thesis. It’s crazy and I am working like a madman, but my person is back and I take pride in my work again.
I’m not really who I want to be yet, and I am still just so ready to move on to the next stage of my life— preferably one with a little more support, and a whole lot more community— but my gears are spinning again. I’m sleeping 6hrs a night again, half of what I was at my worst, but I am so much less tired. My person is not as tired: I feel pretty free, I feel like I have agency, I feel like I’m driving my the car of my life. That was a really, really important place for me to get back to.
I’m not sure how well this all maps onto your experience, but I can see myself pretty clearly in how you describe your state: you feel like you’ve got nothing left. You need a perspective shift. You need to remind yourself what exists outside of this shitty local minima you’re lodged into.
I think my 2 takeaways are that 1) never trick yourself into thinking that any ‘opportunity’ is worth allowing yourself to get to where you’ve got nothing left, and that 2) you can get your person back. I’m not sure what it will take for you, but you should be confident that you can. There’s a lot of stuff that I did over this time that really helped me take control of my life and rebuild my mental state— I got really into playing basketball, making art, etc.— that I didn’t mention, but really, I think the point is to look to your past, identify the parts of your life that have energized you, and start doing those things again. You need some time away from work to get your person back. Remember that a week repairing your relationship with yourself followed by a week of living a more well-adjusted life, where you get a reasonable amount of things done, is much more efficient that two weeks of despair, deriding yourself for not being enough, draining yourself even further.
But anyway— shoot me an email, if you want. I’d be happy to chat and share a little bit more about my experience. I’m still not out of this yet— I’ve got a lot of shit to do before submitting my thesis in 3 weeks— but I did figure out how to recharge those batteries!
I did not seek out a refill for that prescription. Of course, YMMV. But first, make sure your sleep, diet, and exercise are sources of strength, not weakness. Try to unplug & spend a bit more time alone with your mind each day.