I'm a B.Tech student in India graduating in mid-2029. Currently, I know Python, SQL, Docker, basic prompt engineering, and I've built a few LLM apps using BaaS like Supabase/Firebase. I’m running all this on an Intel i5 13th Gen laptop with an RTX 5050 (8GB VRAM). The Pivot: I originally wanted to be a generic ML Engineer, but I've realized the entry-level market for Jupyter Notebook/Python scripters is insanely saturated. Plus, most companies are just calling OpenAI APIs anyway. I want to pivot to the high-performance stuff: AI Backend / Distributed Systems Engineering. Instead of tuning the model, I want to be the guy building the infrastructure that serves the model to 10,000 users without the servers melting. My ultimate goal is top-tier Indian product companies (Zepto, Cred, etc.) or US remote startups. My Proposed Roadmap: I am dropping the "easy" BaaS tools and going low-level. Here is my plan for the next couple of years: The Language Shift: Dropping Python for backend, going all-in on Go (Golang) to master concurrency and memory pointers. The Database Shift: Moving away from Supabase to raw PostgreSQL (via Docker), eventually learning to scale it, plus Redis for caching. The Hardware/AI Constraint: Running models locally on my RTX 5050 using Ollama/vLLM. Learning how to deal with VRAM limits using quantization, PagedAttention, etc., before eventually moving to cloud AWS/GCP. The Distributed Scale: Learning Kafka/RabbitMQ, Vector Databases (Milvus/Qdrant) at scale, and eventually wrapping it all in Kubernetes. Basically, I want to build custom, high-throughput AI backends from scratch. My Questions for the Seniors: Is this roadmap actually viable for a 2029 grad, or is the learning curve for Distributed Systems + AI Infrastructure too brutal to do alongside a college degree? Is betting heavily on Go the right move for this specific AI systems niche? What blind spots am I missing here? Roast my plan. Be brutally honest! submitted by /u/SinkClassic4450 [link] [Kommentare]
Structured primary keys prevent contradiction and perform better than surrogate keys
Inference Cards Jun 25, 2026 Why skip past the why When someone says “I run Qwen 3.6 at 25 tokens per second”, or makes any similar performance claim about their self-hosted LLM setup, this is only meaningful if we know several other things. Which model variant? Qwen 3.6 could be the dense 27B or the 35B-A3B MoE, totally different architectures. Better, just link to the repo you downloaded the weights from. Which quantization? Q8, Q4_K_XL, and IQ3_XXS are at different points in quality/speed/size space. What hardware and inference engine? Is this vLLM on an H100 or llama.cpp on a Raspberry Pi? I could ask if you’re using speculative decoding or various weird stuff, but better, show the command you used to invoke the inference engine. How did you measure the speed? Are you exercising an HTTP API (which adds a possibly-large chat template in the context window), or using something like llama-bench (which skips the template and HTTP/network delay)? Or better, what command did you use to run the test? Also, knowing only the generation (i.e. decode) speed at a shallow context depth is not enough to understand whether agentic workloads will be usable on a given setup. Prefill (i.e. prompt processing) speed matters because agents spend a lot of time reading stuff. It also matters how speeds change as context depth increases, because agents do most of their work with tens of thousands of tokens in the context window. Also, if you’re trying to serve multi-agent (or multi-user) workloads, it matters how these numbers change with multiple concurrent requests. (And no, you cannot guesstimate any of these other numbers from “25 TPS generation speed” because different hardware and inference engines all have different performance characteristics in this several-dimensional space.) With this fuller picture, we can more reasonably compare your computer to my computer. We can talk about which workloads are usable interactively and which will crawl at “run overnight” speed. We can spot when something is broken, and reasonably ask “Does this change make it faster?”, knowing what “it” even was to begin with. We also get a sense of what quality of output to expect from the LLM. In online communities for self-hosted inference, most people don’t bother to communicate most of this information, and the quality of discussion suffers! We need a compact, easy way to share so that more people will do it. Now I follow in some big footsteps to propose a deliberately under-specified plaintext markup format. I hope it is highly readable and easy for new people to pick up. Inference Cards Think of baseball cards, but for computers running LLMs. An inference card shows the most important information to understand setup and performance. You can share them in a code block, or as a screenshot if you hate searching / accessibility. Put inference cards in your pull requests, reddit posts, or wherever you talk about your LLM life. Here is the the world’s first inference card, for my own slop machine. +----------------Inference Card v1-----------------+ | Who+when: cmart.blog, 2026-06-25 | | Weights repo: hf.co/unsloth/Qwen3.6-27B-GGUF | | Quantization: UD-Q4_K_XL | | Platform: Thinkpad T480, Debian 13, eGPU dock | | Accelerator+mem: AMD Radeon AI Pro 9700, 32 GB | | Engine+ver: llama.cpp b9733 | | GPU runtime+ver: ROCm 7.2.4 | |----------------------Tok/s-----------------------| | Concurrency | Context depth | | ↓ | Stage | Empty | 4096 | 16384 | 65536 | |----|---------|--------|--------|--------|--------| | 1 | prefill | 667 | 669 | 640 | 474 | | 1 | decode | 32.1 | 24.8 | 26.6 | 22.9 | | 2 | prefill | 519 | 588 | | | | 2 | decode | 23.3 | 16.2 | | | | 4 | prefill | 526 | 537 | | | | 4 | decode | 16.4 | 9.80 | | | +------------------Config / Notes------------------+ Serving with: ./llama-server \ --hf-repo unsloth/Qwen3.6-27B-MTP-GGUF:UD-Q4_K_XL \ --gpu-layers all \ --spec-type draft-mtp \ --spec-draft-n-max 4 \ --chat-template-file ~/Qwen-Fixed-Chat-Templates/chat_template.jinja Measuring with: uv run llama-benchy \ --base-url http://localhost:8080/v1 \ --api-key "" \ --model "unsloth/Qwen3.6-27B-MTP-GGUF:UD-Q4_K_XL" \ --tokenizer "Qwen/Qwen3-32B" \ --pp 4096 \ --tg 128 \ --depth 0 4096 16384 65536 \ --concurrency 1 2 4 \ --runs 1 \ --latency-mode generation GPU is under-volted with increased power cap via https://github.com/kyuz0/amd-r9700-vllm-toolboxes/blob/main/TUNING.md +----------------End Config / Notes----------------+ FAQ How do you make an inference card? You copy mine from this page and edit the fields. If you hate overtype mode, paste my card into your LLM and ask it to fill in your details. You’re using, e.g., a fork of vLLM? Then specify the repo URL and commit hash instead of the release version. You ran out of space on the card? Add another line or make the card wider. There are no rules. The config / notes section is just free text. Include whatever someone needs to know in order to make sense of your setup, perhaps starting with the commands you use to run it. Should someone be able to read an inference card, buy some hardware, and reproduce the numbers on the card? Ideally yes, but in practice, maybe not. Reproducibility is still a hard problem in high-performance computing. Without loss of generality: what’s the temperature of the air that your GPU is sucking in to cool itself? That affects performance but I’m not expecting folks to measure it. Inference cards are a low rung on the reproducibility ladder. Can someone mislead or lie on their inference card? Of course, it’s the internet! Each of us is only as good as our word. The speed table is a parameter sweep across concurrency and context depth. If your setup doesn’t handle concurrent requests well, maybe don’t include any lines for concurrency greater than 1. Should this be, e.g., JSON? Sure, maybe, but that’s less readable. This is a sloppy first attempt. If you feel inspired to fork and improve this idea, or build tooling around it, please do. Is “Inference Card” a confusing name because people refer to GPUs as ‘cards’? Maybe, but the phrase is semi-novel. I welcome any feedback or complaints. Send them to inference-cards at cmart dot blog. Heck, send me your inference card. This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
I am building a perception pipeline from RGB-D for pick-and-place with a depth camera. The goal is to detect an object on a table, estimate its 3D position, and obtain a stable orientation that the robot can use to grasp it. In this post, I describe some things I learned during the first part of the implementation, after training the segmentation model: https://medium.com/@danieldoradotalaveron/getting-stable-yaw-for-robotics-pca-tabletop-and-the-beginning-of-my-tracking-pipeline-8b9dd3921d3a submitted by /u/nettrotten [link] [Kommentare]
For traders in the EU, there's been a lot of chatter about MiCA and its impact on crypto exchanges. A lot of exchanges are either limiting what they offer to European users or pulling out of the region entirely. This puts a lot of capital at risk and users need to be proactive about it. For those who don't know me, I work in web3 and write content on projects and what's happening in the space. The exchange Bitpanda is running a campaign that fits this moment and helping out users who may be looking to switch exchanges. They're one of the longer-standing European platforms that's fully regulated and stayed compliant through the MiCA transition, and right now they'll pay you to move assets over. The campaign is called Bring Your Assets. Basically, you transfer your crypto into a Bitpanda account, leave it 30 days, and get 5% back paid in Bitcoin. Bitpanda is a regulated custodial exchange, so while you won't be able to do anything onchain, if you're a "buy-and-hold" type of investor and want the benefits of a CEX, this is a nice little boost. If you keep everything in self-custody and never touch an exchange, this isn't aimed at you. Where it makes sense is if you already hold balances on exchanges, or your current European venue is one of the ones cutting back, and you'd rather consolidate onto something regulated that isn't going anywhere. Effectively, you're getting paid 5% to move assets. Some details: The campaign runs from 18 June to 5 July 2026. You transfer crypto into your Bitpanda account through the app or website during that window, and it needs to stay there for 30 days. Everyone who qualifies gets 5% cashback paid in Bitcoin on the amount they bring over. The first 1,000 participants who move €1,000 or equivalent lock in €50 in BTC, which is that same 5% shown as a round number. Cashback is credited by 30 July 2026. It's open to fully verified users in selected European countries, with the full list on the campaign page (https://www.bitpanda.com/en/campaigns/bya-june-26). MiCA is going to keep separating the platforms that did the regulatory work from the ones that didn't, and EU users are going to keep consolidating toward the compliant ones. Bitpanda paying to accelerate that is a smart use of the moment. If you were already going to hold on an exchange, 5% in BTC to pick a regulated one is a reasonable deal. Note: While I did my due diligence and there's no obvious risk with Bitpanda collapsing, I obviously can't guarantee that. Please make sure you are familiar with the risks associated with custodial accounts. Bitpanda has been around for quite some time and has never had any issues. DYOR & NFA - this post is for informational purposes. submitted by /u/TimmyXBT [link] [Kommentare]
While experimenting with GRPO training, I kept running this shit that when reward increases, it becomes difficult to tell whether the policy is genuinely improving or simply exploiting the reward function. So I built a small library called rewardspy that wraps an existing reward function and continuously monitors indicators that often precede reward hacking. It currently tracks things like rolling reward statistics, reward variance collapse, reward component imbalance, response length drift, reward slope changes, GRPO group collapse, anol. This is my first major RL project so I would absolutely love some technical advice Check it out here: https://github.com/AvAdiii/rewardspy submitted by /u/BaniyanChor [link] [Kommentare]
I really don’t get how people trusted a snake with their savings. It boggles my mind. It’s not like he didn’t have a reputation before and he was just some new guy in the space where maybe you could’ve given him the benefit of the doubt. In 2000, he took part in an accounting scandal for MicroStrategy and got charged for it by the SEC. STRC listed 11 percent returns through dividends for STRC that could pause dividends at any time. I know this isn’t the same but Anchor offered around a 20 percent APY in May 2022 for Luna. We all see how that went down. People were confident and sure that this yield could go on forever. I am not claiming it’ll collapse at this exact moment and won’t pretend to know the timing. However, there are so many signs pointing that this will end catastrophically. They own 4 percent of all Bitcoin in existence. People keep saying “they can just sell their Bitcoin.” If they are over-leveraged, the amount of cash and Bitcoin they have won’t matter. They’ll crash at some point and maybe those of you holding might get a dividend payment but don’t get caught with your pants down. At this point it’s a rite of passage for scammers to be on the cover of Forbes. For example, look at Sam Bankman-Fried and Elizabeth Holmes. The signs are there. Those of you invested feel free to call me whatever you want but we’ll see who’s right in the end. Just don’t pretend the signs weren’t there. Best of luck to all of you and I hope you don’t get burned badly. Edit: Those of you invested in MSTR and STRC feel free to talk your shit but I know I’m right and your post will age like milk. Yes I’m petty. submitted by /u/Repturtle [link] [Kommentare]
To make our library for efficient pangenomics more palatable, I made a little Unix shell. It “cheats” when it sees commands that invoke other pangenomic CLI tools and runs built-in functionality instead. The shell is built around an instruction-based IR that can fluidly intermix shell-like I/O (files and pipes) with efficient in-memory data structures. In a debatably fair comparison, the fake shell runs one shell script 48× faster than running it with sh.
