2026s on Ju Lin's AI Weblog

AI Engineering SKill Map

Sat, 22 Aug 2026 00:00:00 +0000

Andrew Ng’s wrote an AI Engineering Skills Map, which lists six things to learn: LLM foundations, grounding models with data, building agentic systems, evaluation-driven development, operating in production, and machine learning foundations.

As he broke it down, I think the most important skill is learning how to build reliable systems from LLM’s uncertain behavior.

You don’t know in advance what an LLM will output

This is the only only truth you need to take away in this post, if you can’t remember all.

ELI5

Sat, 22 Aug 2026 00:00:00 +0000

Thariq at Anthropic posted that it’s a skill people there have been using a lot recently: /eli5 <what you want to explain>.

I tried it on neural networks. Here’s what it made.

The first pass had bad colors, low contrast, hard to read. One follow-up prompt asking for a color fix and it was done.

No formulas, no sigmoid, no softmax, none of the math that actually makes a neural network work. It’s missing for good. What’s left is the dataflow — inputs go in, get combined, come out the other end as a decision — drawn simply enough to follow at a glance.

Pretraining a Mini Kimi K3 for $252

Fri, 21 Aug 2026 00:00:00 +0000

Vizuara AI Labs trained a miniature Kimi K3 from scratch: 1.02B parameters, 145M active, 5B tokens, one H200, $252.35.

Not simplifying the architecture like Karpathy’s microgpt, they kept Kimi K3’s MoE and attention design intact.

That’s a surprisingly cheap way to learn pretraining (in real-world). They worked through expert collapse, data-mixing bugs, distributed-training bugs, kernels, and GPU utilization on a modern MoE architecture.

A few things worth noting:

5B tokens is probably too little for a 1B model. The authors agree the run was budget constrained. So the cheap cost might due to the training stopped early.
Beating GPT-2 isn’t particularly meaningful when Mini K3 has roughly 10× the parameters.
MoE at this scale is debatable. A smaller dense model trained on more tokens would likely be better if the goal was capability.