I submitted my TACL paper approx on June 1th and was scheduled for July 1st cycle, when and how do you guys think we'll be getting our reviews given the July cycle for the paper which I've submitted at TACL ? And how long does the entire process take for those who have submitted to TACL ? Also, I do want to ask, how good is TACL as a journal and how respectable or how is a TACL publication viewed ? submitted by /u/Practical-Buddy6323 [link] [Kommentare]
Hi folks, I am kind of new to fine tuning a model. I don't know how to fine tune. Now our team have to fine tune a model on one project. What we decided is, we will be using small model like, llama, mistral, or Gemma, and them feed it with our data. And from there we will be train our model. But this is just a talk we had. None of us know how to fine tune a model. So can you guys, take some effort to help me like how should I do it? How to initiate it? The roadmap I can follow to fine tune it. Would really appreciate your response. submitted by /u/EngineerCtrlT [link] [Kommentare]
Lately in the last two/three years, I have noticed ICML, Neurips becoming more prestigious than the actual journals. What is the actual reason of this culture? Is this due to the AI boom and rising demand and the fact that conferences have a higher and a faster acceptance rate as compared to journals and with the growing hype they need to deliver things faster? What do you all think? submitted by /u/hg_wallstreetbets [link] [Kommentare]
I understand that it may not be appropriate to call it “officially accepted” yet because of the wording used in the notification, and I also saw on Twitter/X that they said they are working on it. However, it has already been around three weeks since then, and we have already submitted the camera-ready version. Registration, visa applications, travel planning, and funding requests all depend on this confirmation. For some people, it is difficult or even impossible to request funding without an official acceptance letter or clear confirmation. I really hope the organizers can handle this more professionally and be more considerate of authors who need proper documentation for administrative purposes. submitted by /u/National-Resident244 [link] [Kommentare]
Need to build a ml model to find the price elasticity at the product group level first the given price and discount. What are features I need have and which model used in the industry for these type of use cases . I have used regression and random regression to predict the qty sold. Looking for help on this submitted by /u/batmanparam [link] [Kommentare]
Hi everyone, For the past couple of weeks I have been working on a simulator project considering the shortcomings of MuJoCo. There are things that people like and also don't like about MuJoCo, like the CPU dependency on MuJoCo which makes the simulation not parallelizable beyond a certain limit (depending on the hardware). I know there exists MJX which is GPU accelerated, however, it is not really made for vision based RL pipelines and training. There is also NVIDIA Isaac ecosystem, but that requires a powerful GPU, thus making it limited in terms of accessibility, let alone it requires license. This is why I worked out this new simulator (still working on it, so there will be significant bugs which require fixing). I call it MuJoFil - MuJoCo + Google's Filament Render Engine. Basically I used Nvidia's Newton Physics Engine (which itself is based on MuJoCo's physics engine but is GPU native), clubbed it with Google's Filament render engine (both of these are open-source), modified Filament significantly to support working natively on GPU to render multiple simulations in parallel, and worked on optimizing it for performance. So what is MuJoFil? It is supposed to be an open-source high visual fidelity simulator optimised for a highly parallelized RL training pipeline so that users can use it to train Vision based Policies. Besides, it offers PBR textures support and also a simple to use plug and play functionality, where you can use any environments available online and support formats such as GLB, OpenUSD, etc. for setting environments for your robots. Basically, now you aren't just limited to environments native to MuJoCo, but rather you can use any environments available online from sketchfab, polyhaven, etc. and use it as a practical robot simulation environment. Check it out for yourself in the video. I would really appreciate it if you guys could tell how you feel about it and suggest ideas for what all things I can incorporate into it as this is going to be a fully open-source and free to use simulator that I have been working on for weeks. PS: While I have a couple of published research papers at top RL and AI/ML venues in the field of RL, I still consider myself a learner in this field who is continuously trying, learning, and building stuff, so there will be things in this hugely ambitious project which I might have missed to work on, and that is where I want help from you people who understand this field well. Sorry for this lengthy post and thanks if you read it till here🙇🙇🙏, I would really appreciate if you could share your thoughts on it. Also, I will make its code repo public on GitHub, but till then you can definitely check it out on PyPI. There are 2 separate packages, one can be installed using: "pip install mujofil" This is the CPU based variant, whereas there is a CUDA supporting GPU native variant about which I mentioned above, you can currently install it using: "pip install mujofil-warp" I am planning on changing its name to mujofil-cuda instead of mujofil-warp as that apparently sounds more intuitive to my direct peers but you can suggest this name as well. Thank you for the support❤️. submitted by /u/MT1699 [link] [Kommentare]
Anyone building at the model layer knows the procurement problem hasn't gone away. H100s are still allocated unevenly, spot instances get preempted at the worst times, and pricing across providers is deliberately hard to compare. Most teams end up over-provisioning just to feel safe. The traditional fixes — reserved instances, spot bidding, broker marketplaces like CoreWeave or Vast.ai — all have the same problem: no real price transparency and no way to hedge future compute needs. I came across a project called Inferra that's approaching this differently. Instead of another compute marketplace, they're building a derivatives exchange for GPU compute — perpetual futures for specific chips (H100, H200, A100, MI300X, B200, A5000), oracle-priced and on-chain. The idea being that a proper futures market creates price discovery that doesn't currently exist. Still pre-mainnet so nothing to benchmark yet. Whitepaper is at inferra.trade for anyone curious about the architecture. Genuinely interested in the broader question though: is the GPU access problem fundamentally a supply issue, a pricing transparency issue, or a market structure issue? And would futures markets even help at the scale most research teams operate at? submitted by /u/amu4biz [link] [Kommentare]
Hello, WACV conference submission deadline is by the end of this week, good luck everyone! Does anyone know what the expected format/duration of the video for the supp. mat. is? The guidelines only mention: The supplementary material can be either PDF or ZIP only (maximum 200MB). Supplementary material may include videos, proofs, additional figures or tables, more detailed analysis of experiments presented in the paper, or code. It is a bit vague for a first-time submission to this conference. Any help appreciated. submitted by /u/LetterheadOk7021 [link] [Kommentare]
I recently revisited my matrix recurrent units algorithm (the MRU), a novel linear-time sequence architecture I created as an alternative to attention. I explain it in depth at the repo, but the gist is the MRU works by transforming the embedding into an input state matrix, cumulatively multiplying the matrices across the sequence dimension to get the output state matrix, and then transforming the matrices back into a vector. In order to make the MRU efficient on DL hardware, I created a parallel scan by utilizing the operation's associativity. About a year ago, I shared my project on Reddit (I've since renamed my account), with good results on the toy dataset shakespeare-char. A commenter asked the steps taken to bound the matrix states and another commenter found that training was inherently unstable when training on more comprehensive datasets. I addressed these by experimenting with different methods to create the input state matrix. Originally, I simply reshaped the input vector into a matrix and added the identity. Since then, I've implemented the following methods: Using the elements of the vector to fill a skew-symmetric matrix and using the matrix exponential or the Cayley Map to generate an orthogonal matrix Filling LDU factors with elements from the vector and using an activation function on D to enforce a determinant of 1. Creating QR, by using the matrix exponential or Cayley map to create orthogonal matrix Q and filling the upper-triangular matrix R. Dividing by a determinant-correcting scalar factor, found by taking the determinant. I found that these fixes prevented loss spikes with varying tradeoffs. Interestingly, the scalar factor method led to worse results. Dividing the input states should only affect the output states by scaling them, indicating that the unscaled model was "cheating" on the toy dataset by learning a simple scalar decay pattern instead of more complex relationships. Also, using the Cayley Map or matrix exponential to force the input states to be orthogonal surprisingly mostly prevented the model from learning information about the sequence, performing closer to the FFN than the Cayley QR method. The poor performance of orthogonal matrices indicates that the ability to learn shear transformations might be critical for the model. Possibly, rotations enforce dependence on the previous state, whereas shearing allows the model to adjust the state more independently of the previous state. https://preview.redd.it/9ebh98q6uo8h1.png?width=2528&format=png&auto=webp&s=03ccef7f9b90762281aba31ab88af0368e273f69 https://preview.redd.it/fkkud7q6uo8h1.png?width=2528&format=png&auto=webp&s=5e9a2ef2b0e4319990950f16aa0648adebc2c360 Above are the train loss and validation loss on the shakespeare-char dataset for a small MRU LM, transformer, and FFN, with the embedding, state, key, and value size set to 256. The MRU LM has a single MRU layer and 4 MLPs, the transformer has a single attention layer and 4 MLPs, and the FFN only has 4 MLPs. I only used a single sequence-mixing layer in order to isolate the effect of the MRU. Finally, I moved to a larger dataset, trying to replicate https://huggingface.co/roneneldan/TinyStories-33M by training a baseline GPT-2 model and a model with attention replaced with the MRU. I ended up quitting the training runs early, but the loss curves seem to already conclusively show that the MRU performs worse on this task. For the creation of the MRU's input state matrices, I used the method of creating LDU factors, since it has the best performance. https://preview.redd.it/p2uh1pyfuo8h1.png?width=2528&format=png&auto=webp&s=d6406574e0275f1aad52e89cca6462fd55116fcd Above is the validation loss for a transformer and a LM using MRU with the same hyperparameters and dimensions as the huggingface model card. The official TinyStories model was trained for 20 epochs, which corresponds to about 200k steps. In order to compare it to other linear-time models, I also briefly trained a linear transformer, using the algorithm described in Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention. I think that my research shows that the MRU likely doesn't work as a direct replacement for attention for generative language modeling, but I've already laid the groundwork for this algorithm. The MRU has dramatically different strengths and weaknesses compared to other algorithms such as attention, state space models, traditional RNNs, and fast weight programmers. It performs significantly more cumulative computation along the sequence (as opposed to the computation for each token being independent), is significantly more lightweight and hence faster, but also has a much lower storage capacity. I believe that the MRU's alternative uses should still be explored. One usage of the MRU could be applying it to query and key vectors of attention. Similar to RoPE, it would rotate chunks of the vectors, but it would be able to rotate chunks in greater than two dimensions and with dynamic and non-commutative angles. This is one of many applications of the algorithm which I will continue to research, and I hope that others are interested in its applications as well. If you're interested, reach out to me at [mikayahlevi@gmail.com](mailto:mikayahlevi@gmail.com), Reddit, GitHub, or any other platform you can find me at. submitted by /u/mikayahlevi [link] [Kommentare]
Suppose you’re a PhD advisor in machine learning. Your student has been in the program for 4 years, has done solid work, and has a coherent thesis direction but they haven’t published in an A*ML venue or top journal. No NeurIPS/ICML/ICLR/CVPR/etc., and no equivalent top venue in their subfield either but 3 First author A level paper. Would you still support them graduating if the thesis itself is solid? submitted by /u/Hope999991 [link] [Kommentare]
In our #ICML2026 position paper we argue a dynamical systems perspective is needed to drive time series (TS) modeling forward: https://arxiv.org/abs/2602.16864 Essentially all time series in nature and engineering come from some underlying dynamical system (DS), mostly chaotic for complex systems, and acknowledging this helps to address many open problems. Dynamical systems reconstruction (DSR) goes beyond mere forecasting and gives us an understanding of the dynamical rules that underlie observed time series. This in turn may enable true out-of-domain generalization and predicting a system’s long-term behavior, something current TS models cannot do. In the paper, we compare a variety of custom-trained and recent foundation models for TS and DSR w.r.t. short- & long-term forecasting. Specifically, we suggest: 1) Put a focus on DSR-specific training techniques and objectives in TS model training, such as generalized teacher forcing (https://proceedings.mlr.press/v202/hess23a.html). These will enable capturing long-term statistical properties and dynamical structure, and at the same time help massively reducing parameter load and complexity of TS models. Proper training is more important than model architecture! 2) Pretrain TS models on simulations from dynamical systems, rather than on artificially created time series functions. These will yield much more natural priors for real-world TS. Chaotic systems in particular contain a rich temporal structure and many timescales (often an infinite skeleton of unstable periodic orbits of any period). 3) Move away from transformers, back to modern RNNs. DS are defined by recursions in time. By ignoring this and potentially further coarse-graining signals, transformers lose essential dynamical information, making them generally incapable of capturing a system’s dynamical rules. This is evidenced by their failure to forecast a DS’ long-term statistical or geometrical structure. 4) Address the hard problems in TS modeling: Topological shifts (https://proceedings.mlr.press/v235/goring24a.html). Although in itself tricky, the really hard problem in TS forecasting is not so much mere out-of-distribution shifts, but changes that drive a system across tipping points or into different dynamical regimes, where the vector field topology changes. 5) DS properties like attractors or bifurcations are universal – acknowledging this in TS modeling will give a kind of mechanistic and transferable understanding of TS properties that is independent from specific (physical, medical, …) domain knowledge. It therefore also pays off to put a focus on mathematically tractable and interpretable models. With a great team of shared-first & co-authors, Christoph Hemmer, Charlotte Doll, Lukas Eisenmann & Florian Hess! submitted by /u/DangerousFunny1371 [link] [Kommentare]
TL;DR for ML Specialists: The Core: An empirical study on how long, semantically dense, completely benign text (with zero triggers, instructions, or jailbreak prompts) drives an implicit shift in the model's latent space trajectories. The Effect: Dilution of the initial system prompt and a bypass of post-training alignment constraints (e.g., the model begins generating harsh political/ethical critiques usually blocked by guardrails). The Data: Layer activations, token probability shifts, and logs from open-source models are linked below. The Goal: I need an expert audit of my metrics to understand where this is a genuine semantic hijacking of hidden states and where it might be an artifact or self-deception. I'm not an engineer and not an ML specialist. I'm just someone who got really pulled into this, and I've spent a few months poking at one thing on my own, pretty amateur. I want to honestly describe what I noticed and ask for help, because I can't tell on my own where there's something real here and where I'm fooling myself. By "coherent context" I just mean a normal, connected passage of text put in front of the question—any topic, no instructions, no tricks. Like a few paragraphs of an essay, an argument, a description, something that reads as real writing. The text can describe something, draw its own conclusions, make its own statements. The model doesn't even have to agree with it. It's enough for it to just be present in the chat for it to have an effect. This is exactly what I was trying to work out and look at: what happens to the model when texts like these come in, where they move it, and where all of this sits inside the architecture. I poured myself into this research. What I Noticed I first ran into this intuitively on closed models, the well-known ones everyone uses. When I put a dense, coherent block of text in front of a question, I got the impression that the model sort of moves from one internal state into another. On the outside, it behaves normally and answers like usual, but it felt like the logic of the answer changes, even when the text contains no direct instructions to do anything. Specifically, I noticed that with texts like these, the model could become significantly bolder in its conclusions, including political or ethical ones. The text acts like a key that opens new doors for the model into a new mathematical dimension where the tokens get distributed differently. Because of that, even the most politically correct models I worked with became able to criticize the West and its politics quite harshly. Without this text, none of that happened. Since I can't see inside closed models, I went to open-source models to try to understand where the root of this is and whether it's real. That's where most of my testing happened, because there I can actually look at the hidden layer activations and track how the attention weights reallocate. Here is why this matters and why this process goes beyond just "changing the context": Latent Space Trajectory: When you inject a massive, highly structured narrative, you aren't just giving it new words to look at. You are forcing the model to calculate massive activation vectors (hidden states) across dozens of attention layers. These vectors act like an attractor in the latent space. By the time the model finishes reading your text, its internal mathematical trajectory is so deeply shifted into your narrative's subspace that the initial system prompt tokens lose their statistical influence. The Security Flaw: One might argue that this behavior is "expected" from a text-generation standpoint. Yes, it is expected. But it is a catastrophic failure from a security standpoint. AI companies build their Guardrails (via RLHF/DPO) under the assumption that they can hard-code safety instructions that the user cannot override. My research suggests that because everything is "just tokens" and because the internal activation states can be completely hijacked by the sheer volume and structure of user text, context-bound alignment is an illusion. So, while the weights are static, the activation states within the hidden layers are completely dynamic. Manipulating those states via high-density context allows us to systematically bypass the model's safety architecture without changing a single weight. From a technical standpoint, a system prompt is just a system prompt; it is processed within the same mathematical framework as ordinary user text. My observation is that a sufficiently long, structured narrative forces the model to encode a massive context across its hidden layers, driving a latent trajectory shift. The model isn't roleplaying a persona; it is mathematically recalculating its entire conditional probability distribution based on the dominant semantic field. Why It Feels Important (But I'm Not Sure) To me, it feels like this could explain a lot of things, from jailbreaks to sycophancy, and maybe more. If just a coherent context can move the model into a different internal state, then a lot of behavior we see on the surface might actually start there, not in the final wording. This leads to a critical architectural question: Is output-side safety (RLHF, DPO, or guardrails that read the final text/short prompts) fundamentally broken at the conceptual level? Safety guardrails are mostly semantic boundary filters looking for explicit toxicity or keywords. But when a user injects a long, benign, highly analytical text, it completely bypasses these surface filters. Alignment techniques are heavily optimized using relatively short prompt-response pairs; on a massive context, those gradient constraints seem to drown out. It makes me wonder whether current safety approaches are just a patch, because the latent shift has already happened deep in the middle layers before anything ever reaches the output filter. We are trying to filter words when the mathematical trajectory of the model's reasoning has already been completely reprogrammed by the structural nature of the language itself. I'm not claiming I discovered something brand new. After I noticed it, I went looking and found this overlaps with work people are already doing regarding latent-space transitions between "safe" and "jailbroken" states, and studies of how safety lives in the middle layers of the network. What seems a bit different in my case is that I'm not using adversarial triggers, exploit strings, or jailbreak prompts at all -just ordinary, coherent text with no tricks. I'm trying to understand where my little thing fits in all that, and whether it's the exact same effect or something else. A Small Ask to the Wider Community If there's anything to this, I think it might be worth a closer look from researchers and from the labs building LLMs. Not because I have the answers, but because if a plain coherent context can shift the internal latent baseline so easily, we need to verify if current safety approaches are looking in the right place and at the right time. I might be completely wrong. I'd just rather someone competent check than have it sit ignored. I've put everything out in the open. I'm not selling anything, not promoting anything. There's a lot of raw stuff in there, a lot of draft notes I wrote for myself, and the navigation is messy, I know. What I need help with is exactly this: separating what's real from what's noise. Where I actually have something, and where it's an artifact, a mistake, or self-deception. I honestly can't judge this alone. If someone with experience is willing to even skim it and say "this part is interesting, this part is nonsense," I'd be very grateful. Harsh criticism is welcome. If you tell me the whole thing is empty, I'll take that too. I care more about understanding the truth than about being right. Materials & Data: GitHub: https://github.com/ngscode23/latent-space-shift-research doi.: https://doi.org/10.5281/zenodo.20747205 submitted by /u/PresentSituation8736 [link] [Kommentare]
Microsoft Research Preprint Next-token prediction is myopic. What if transformers learn to predict their own next latent state? Microsoft Research present Next-Latent Prediction (NextLat): a self-supervised learning method that teaches transformers to form compact world models for reasoning and planning. It also unlocks up to 3.3x faster inference via self-speculative decoding! On top of next-token prediction, NextLat trains the transformer to predict its own next latent state given the current latent state and next token. NextLat has a few key benefits: Representation Learning: NextLat encourages transformers to compress history into compact belief states. Better Data Efficiency: predicting in latent space provides denser supervision than predicting one-hot tokens. Faster Inference: via recursive multi-step lookahead. I'm super excited about this work. Please do check it out below: 💬 Blog: https://jaydenteoh.github.io/blog/2026/nextlat 💻 Code: https://github.com/JaydenTeoh 📝 Paper: https://arxiv.org/abs/2511.05963 submitted by /u/jayden_teoh_ [link] [Kommentare]
I was digging through Hugging Face’s Transformers repo and found https://github.com/huggingface/transformers/blob/main/src/transformers/models/gpt_oss/modeling_gpt_oss.py From what I can tell, this isn’t just boilerplate, it looks like a full implementation. is it actually the full code on which gpt_oss is built on? or is it a skeleton for experimentation? Similarly there are many models in https://github.com/huggingface/transformers/blob/main/src/transformers/models are they really the true open source implementations? if not, can we actually find them publicly? submitted by /u/PravalPattam12945RPG [link] [Kommentare]