I ran three prompts against DeepSeek V4 Flash 0731 on a DGX Spark sitting in my office. I ran the same three prompts against DeepSeek's own hosted service in Expert Mode. Then I opened the six resulting HTML files, looked at them, and picked a winner for each.
Two of my three calls were wrong. In both cases the thing that made an output attractive to me was a defect in its code, not a decision the model made.
That is the finding. The rest of this post is the evidence, and then an honest accounting of what this comparison does and does not measure.
Set your expectations first: this was a quick test. Three prompts, one generation each, run in an afternoon. It is not a benchmark and I am not presenting it as one. It is the kind of thing anyone can do in an hour to get a feel for a model, which is exactly why it is worth writing up when it turns out that doing it casually produces the wrong answer.
Setup
On-premises DeepSeek-V4-Flash-0731
Hardware NVIDIA DGX Spark, GB10, 128GB unified LPDDR5X
Serving Single Spark, same build as our DeepSeek V4 Flash DGX Spark build
Reasoning effort Max
Hosted chat.deepseek.com, Expert Mode
Reasoning effort Max
Generations One per prompt per side. No retries.
Repair rounds None.
Prompts Pasted verbatim, identical text to both sides.
Output Single self-contained HTML file, opened directly in Chrome.
Both sides ran at maximum reasoning effort, so the most obvious confound is off the table. That matters, because DeepSeek's own technical report notes that both V4 models improve on knowledge benchmarks when given more thinking budget. Had I run the local side at a lower effort level, everything below would have been a thinking-budget result dressed up as a deployment result.
One thing I cannot tell you: exactly which checkpoint the hosted side ran. Expert Mode is a web interface mode, not a model identifier, and DeepSeek's changelog for the 0731 release states that the update covered the V4-Flash API only, leaving the app and web models unchanged. My guess is some version of V4. I would not stake anything on which one.
That is a real limitation and I am not going to dress it up. Next time I will run the hosted side through an inference provider with a pinned model ID so there is no ambiguity about what answered. This one was a quick test, not a study.
Both sides are DeepSeek. This is not an open-weight model against a frontier API. It is one vendor's model on my hardware against the same vendor's model on theirs.
Round one: DNA
generate detailed 3d of a dna, show it html
Drag to rotate, scroll to zoom. Each demo runs a live WebGL context, so unload one before loading the next if the page slows down.
I preferred the on-premises version immediately. Denser, higher contrast, stronger silhouette. The hosted one looked thin next to it.
Then I read the code.
The on-premises version creates one material object outside the rung loop, sets baseMat.color inside the loop, and hands that same object to all 106 cylinders. Three.js materials are shared by reference. Every rung takes whatever colour was assigned last. The colour variation the code appears to implement does not exist. Every rung renders orange.
The spacing is wrong too. 106 rungs across 9.8 units of height gives 0.092 units of gap, against cylinders 0.12 wide. The rungs are wider than the space between them, so they fuse into a continuous mass.
What I was looking at was not a ladder between two strands. It was two tubes with a solid orange wedge filling the gap. The density that made it striking was the failure.
The hosted version, meanwhile, had done actual biology. It built a complement map, generated a sequence, derived the second strand, and split each rung into two half cylinders so each base renders against its correct partner. It drew two hydrogen bonds for A to T and three for G to C. Its dimensions were real B-DNA: 0.34 rise per base pair, ten bases per turn, radius near one nanometre.
The prompt asked for detail. In a DNA context, detail means base pairing and hydrogen bonds. The hosted side delivered both. The local side delivered a helix with a decorative colour cycle that encoded nothing and did not render.
The hosted side also handled its render pipeline properly: filmic tone mapping, correct output colour space, fog, and an explicitly sized shadow camera frustum. The local side configured none of that.
It was not clean, though. It allocated duplicate sphere geometries and materials inside a 32-iteration loop, declared two geometries under a "Shared Geometries" comment and never used either, and set a per-particle size attribute that PointsMaterial ignores entirely. Wasteful, but none of it is visible.
Both sides also got the groove structure wrong. Real B-DNA offsets its strands asymmetrically, producing a wide major groove and a narrow minor groove. Both used a flat 180 degree offset and produced two identical grooves.
Round two: voxel garden
Design and create a very creative, elaborate, and detailed voxel art scene of a pagoda in a beautiful garden with trees, including some cherry blossoms. Make the scene impressive and varied and use colorful voxels. Use whatever libraries to get this done but make sure I can paste it all into a single HTML file and open it in Chrome.
Drag to rotate, scroll to zoom. Each demo runs a live WebGL context, so unload one before loading the next if the page slows down.
I preferred the on-premises one again, by a wider margin. Softer palette, rounded voxels with visible gaps, mountains and clouds in the background. It looked art directed. The hosted version looked like default Minecraft.
The soft palette was a bug.
The local version passes 0-to-1 floats to new THREE.Color(r, g, b). Since Three.js r152, colour management is on by default and bare floats are read as already being in the working colour space, which is linear. No conversion happens. The renderer then encodes linear to sRGB on output and everything lifts. A ground colour intended as mid olive green comes out as pale sage.
The confirming detail is in the same file. The sky is set with a hex value, which routes through a different code path with sRGB as the default and converts correctly. The sky renders as a proper saturated blue while the voxels beside it are pastel. Same file, two code paths, only the float path washed out. The local version is also running lower exposure than the hosted one, so it should be darker, not lighter.
The pastel look I read as polished was not chosen. It is what a colour space mismatch looks like when it happens to land somewhere pleasant.
This round the local model earned real wins, and they matter more than the palette.
It built the entire scene as a single InstancedMesh with per-instance colours. One draw call for the whole world. The hosted version grouped voxels by colour and created a separate InstancedMesh per colour, against a palette of roughly sixty entries. Sixty draw calls doing the work of one. The local approach is correct and it is not close.
It also made a real art direction choice: rounded box geometry at 0.9 scale, so each cube has a visible gap and a soft edge. That is deliberate, not a default. And it built background mountains and cloud clusters, so the world has depth behind it. The hosted version's world simply ends at the edge of the grass.
Where it lost is content. The prompt said elaborate, varied, impressive. The hosted side built a torii gate with correctly named components, an arched bridge with railings, a pond with depth-graded water, two zen gardens with alternating raked sand, four stone lanterns with lit chambers, a segmented bamboo grove, three tree species plus bamboo, rock clusters, and flower patches. It animated forty falling petals with per-petal velocity, delta-timed sway, rotation, and respawn on ground contact. The local side built a pagoda, cherry trees, pine trees, stepping stones, bushes, flowers, one lantern, and a small pond, with no animation at all.
I score this one to the hosted side on prompt compliance, narrowly. The local side won the two engineering decisions a graphics engineer would care about most and lost on volume of correct detail.
One more thing I got wrong here. I complained that the hosted version did not fit the screen. It does. My two screenshots were different viewport sizes, and that version auto-rotates, so its framing depends entirely on when I hit capture. That was my capture, not its code.
Round three: the jet
generate detailed 3d boeing jet, show me the code
Drag to rotate, scroll to zoom. Each demo runs a live WebGL context, so unload one before loading the next if the page slows down.
The local side built a 737-800. The hosted side built a 747. Both are valid readings of "Boeing jet."
This time I preferred the local one, and this time I was right, mostly.
It produced far more aircraft: flaps, ailerons, rudder, fin tip, engine duct rings, eighteen fan blades per engine, spinner, exhaust nozzle, navigation lights, tail beacon, and a correctly wired CSS2D label anchored in 3D space with its own render pass and resize handler. The prompt said detailed. It delivered more detail.
It also got the navigation lights right. Red at positive X, green at negative X. With the nose at positive Z and up at positive Y, right-handed coordinates put starboard at negative X. Green on starboard, red on port, which is correct aviation convention.
Its rendering advantage was real and not a bug this time: filmic tone mapping, fog, and a three-point coloured lighting rig against the hosted version's single white directional light plus flat ambient. The grey flatness of the hosted render is the absence of a pipeline, not a style.
Where it lost:
The wingspan is roughly half of what it should be. Full span comes out near 14.4 units against a fuselage running about 32.7 nose to tail, a ratio of 0.44. A real 737-800 is 0.91. That is why it reads as a missile with fins. The hosted 747 lands at 0.89 against a real 0.97.
Shadows are missing for the same reason they were missing in round one. The local side enabled shadow casting and never sized the shadow camera frustum, which defaults to plus or minus five units on a model spanning plus or minus sixteen. Most of the aircraft falls outside it. The hosted side sized its frustum explicitly and got a clean ground shadow. Two rounds out of three, the local side skipped this same step.
The fan animation never runs. The loop grabs child index 1, which is the duct torus, not the fan group at index 2, and the type guard silently fails. Even with the right index it rotates around Y while the engine axis is Z. Two bugs stacked, in the only animation in the file.
The aircraft is also buried. The fuselage belly sits below the ground plane and the plane is not resting on its gear. The hosted version has the mirror problem, with its wheels sinking through the ground instead. Neither landed the aircraft on its own gear.
And the local side called the winglets "sharklets." That is Airbus terminology for the A320 family. On a model labelled BOEING 737-800, anyone who flies will catch it.
The hosted side's domain error is bigger. It named the aircraft a 747, correctly built four engines and the correct 747 landing gear layout of four main bogies plus nose, and then omitted the upper deck hump. That is the single most recognisable feature of a 747. It named a specific aircraft and left out its identity.
What the pattern looks like
Three rounds is not a benchmark. But a shape shows up, and with reasoning effort matched on both sides it is worth taking seriously.
The hosted side won on knowledge and precision every time. B-DNA rise and radius. Base pairing and hydrogen bond counts. The 747 gear configuration. Airframe proportions within a few percent. Breadth of scene content when the prompt asked for variety.
The local side won on pattern-level craft. Tone mapping. Fog. Coloured three-point lighting. Instanced draw calls. Rounded geometry with deliberate spacing. A correctly wired 3D-anchored label. Aviation light conventions.
Same model family, same reasoning effort, different numerical precision. My local build is quantised. The hosted one is not.
If quantisation degrades recall of specific values more than it degrades stylistic pattern matching, this is exactly the split you would expect. Structural habits like reaching for tone mapping or instancing survive. Retrieving that B-DNA rises 3.4 angstroms per base pair, or that a 737-800 has a 0.91 span-to-length ratio, does not.
That is a hypothesis, not a result. Three prompts and one generation each cannot establish it. But it is testable, it is the kind of thing that matters if you are deploying quantised weights on your own hardware, and it is the direction I am taking this next.
Why the screenshots failed me
I run GPU infrastructure for a living. I have spent years around this hardware. I looked at three pairs of screenshots and picked the broken one twice.
Both times the bug was the thing that made the output attractive. Fused rungs read as density. A broken colour space reads as a soft designer palette. Neither was a decision the model made. Had I published on the screenshots alone, I would have argued that a local deployment beat a hosted one on the strength of two rendering defects.
If you are deciding whether an open-weight model on your own hardware is good enough for your workload, looking at the output is the least reliable method available to you. You have to read what it produced, run it, and check it against something that has a right answer.
Benchmarks are a starting point, not an answer
There is a lot of good published evaluation data now. Terminal-Bench, SWE-Bench, the vendor cards, the independent trackers. I read all of it and it is genuinely useful for narrowing a field. It tells you which two or three models are worth your time.
It does not tell you which one is right for you. Nobody publishing a leaderboard knows what you do all day.
Not one of the three prompts in this post appears on any benchmark. There is no eval for "does it know that a 747 has a hump" or "does it reach for tone mapping without being asked." Those mattered to me because generating browser-native 3D is something I actually do. If your work is contract review or claims triage or log analysis, none of what I measured here transfers, and you should not let it.
So run your own prompts. Your real ones, the ones you type on a Tuesday, not the ones designed to make a model look good. Then judge the output against what you personally need it to be, because you are the one who has to live with it.
But judge it properly. This is where my own two wrong calls matter. Taste tells you what to test and what "good" means for your work. It does not tell you what passed. I trusted my eye on three renders and my eye was wrong twice, on artefacts I could have checked in ten minutes by reading the source. Personal judgment picks the criteria. Evidence decides the outcome. Skip the second half and you will end up defending a bug.
This is what I do for clients
Worth saying plainly, since it is the reason I run tests like this at all.
GPUNexus builds private on-premises inference systems. DGX Spark and RTX PRO hardware, frontier open-weight models, running inside your building with no cloud API calls and nothing retained by a third party. For legal, healthcare, finance, and research teams, that last part is usually the whole decision.
The service that matters here is the assessment. We take your actual prompts, not a benchmark suite, and measure them on local hardware: throughput, latency, concurrency, and whether the output holds up when you read it rather than glance at it. This post is a small unpaid version of that, run on my own time with three prompts I made up. The paid version uses yours, runs enough generations to say something, and reads the output against a rubric written down in advance.
If you want to try the models first, there is a thirty minute session on a private endpoint. Bring three real prompts from your work. That is enough to learn whether this is worth a longer conversation, and it costs you half an hour.
What I would change
Every criticism above applies to my method too.
One generation per prompt is not enough. At temperature 1.0 a single sample tells you almost nothing, and one shared-material bug does not characterise a model. Three to five per prompt, minimum.
No repair round. I never told either side what was wrong and asked it to fix it. That is how anyone actually works, and it tests something one-shot generation does not. A model that ships a bug and finds it on the first correction is more useful in practice than one that ships clean and cannot iterate.
No rubric fixed in advance. I scored after looking, which means the rubric bent toward what I already preferred. Prompt compliance, domain correctness, code correctness, and visual result should have been written down before the first generation.
No standardised capture. Same viewport, auto-rotate off, fixed delay after load, no manual zoom. My framing complaint in round two was entirely my own fault.
What is next
Three things are already in motion.
I am rebuilding this test across two DGX Sparks rather than one. Combined memory changes which models and which precisions are reachable, which goes directly at the quantisation hypothesis above. If the knowledge gap narrows when I stop quantising as hard, that is the answer.
I am adding a genuine third party to the comparison. Fireworks or a comparable inference provider, so the hosted side is not just the model vendor serving its own weights. Right now I have DeepSeek against DeepSeek, which is a clean control but a narrow one.
And I have started running these same prompts against Qwen3.8-27B on the same hardware. Early on it is beating both DeepSeek configurations, which I did not expect from a 27B dense model against a 284B mixture-of-experts.
I am telling you that and telling you in the same breath not to believe it yet. It is one generation per prompt, judged by eye, by someone who has just spent two thousand words demonstrating that he judges these badly by eye. It is exactly the kind of early impression this post argues against. It becomes a result when it survives the method I just described, with a rubric written down first and the code read line by line.
That is the next post.