A shadow test that sampled an icon: testing flat design with pixels in Swift

TL;DR

• A visual bug can be pinned down with a handful of single pixel reads. NotchStar’s ShadowArtifactTests renders the real root view at the real panel size over white, converts a few chosen points to luminance, and asserts a number for each. There is no stored reference image.
• One of those points sat on an icon. When the tab bar changed, the pixel at the horizontal centre of the bar stopped being black and a test about the notch shape failed. The fix was to sample beside the tab bar, at a point no layout change can reach.
• The suite has 5 tests in 91 lines. It cannot run here without an Xcode build, so this post quotes the code and the git history and states what was not run.

The problem: a black block under the notch

NotchStar opens a panel from the notch. The open shape is solid black with a soft shadow around it, so on a light desktop it looks like it floats. The shadow is drawn as a separate layer behind the shape: an opaque copy of the open notch with a blur, rasterized once with drawingGroup() and then faded in and out.

The original version tied that layer’s opacity to the open state, so it faded in as the real notch started to grow. The real notch was still small, while the shadow layer was already a full size opaque black copy with a blur. While the notch grew, the screen showed a black block under it. The commit that fixed it, 969c58b on 7 October 2026, describes the cause in the source comment.

/// Fixed-size layer, rasterised once on the GPU. It is an opaque copy of the *open* notch plus a blur, so it
/// may only be visible once the real notch has finished growing (the controller flips `shadowVisible`
/// after the open animation and clears it the instant a close begins). Otherwise it shows as a black
/// block under the notch.

A bug like this is awkward to guard. A unit test on a boolean can check that shadowVisible is false while the notch is growing, and the suite does that. It cannot check that the screen shows no black pixels. The regression is visual, and only a rendered pixel can say it.

Approach: render, sample, compare to a number

The test file makes three decisions. Render the real view, not a copy. Read a few pixels, not a whole image. Compare each to a threshold with a message that says what went wrong.

Render the real view at the real size

The panel is the open size plus a transparent margin on the left, right and bottom, so the shadow has room.

private let m = NotchModel.shadowMargin
private var panelSize: CGSize {
    CGSize(width: NotchModel.openSize.width + 2 * m, height: NotchModel.openSize.height + m)
}

openSize is 660 by 296 points and shadowMargin is 30, so the panel is 720 by 326 points. The render puts the real NotchRootView over a white background in an NSHostingView, gives it a borderless window, and lets the run loop turn for 0.4 seconds before drawing.

private func render(_ model: NotchModel) -> NSBitmapImageRep {
    let root = NotchRootView(model: model).background(Color.white)
    let host = NSHostingView(rootView: root)
    host.frame = CGRect(origin: .zero, size: panelSize)
    let w = NSWindow(contentRect: host.frame, styleMask: [.borderless], backing: .buffered, defer: false)
    w.contentView = host
    host.layoutSubtreeIfNeeded()
    RunLoop.main.run(until: Date().addingTimeInterval(0.4))
    let rep = host.bitmapImageRepForCachingDisplay(in: host.bounds)!
    host.cacheDisplay(in: host.bounds, to: rep)
    return rep
}

The white background is the point. A shadow is a gradient from black to the background colour. On white, every dark pixel outside the shape is shadow or a bug, so brightness maps directly to the thing under test. Over a transparent background the same pixels would be alpha values, which are harder to reason about.

The window is never ordered on screen. cacheDisplay(in:to:) draws the view into a bitmap from the hosting view.

Luminance at a point

A pixel is turned into one number, so every assertion reads the same way.

private func luminance(_ rep: NSBitmapImageRep, x: CGFloat, y: CGFloat) -> CGFloat {
    let sx = CGFloat(rep.pixelsWide) / panelSize.width, sy = CGFloat(rep.pixelsHigh) / panelSize.height
    guard let c = rep.colorAt(x: Int(x * sx), y: Int(y * sy))?.usingColorSpace(.deviceRGB) else { return -1 }
    return 0.299 * c.redComponent + 0.587 * c.greenComponent + 0.114 * c.blueComponent
}

Two details. The points are in view coordinates with the origin at the top left, and the scale factors turn them into pixels, so the same test works on a 1x and a 2x display. The weights are the usual Rec. 601 luma weights, 0.299, 0.587 and 0.114, which give a value from 0 for black to 1 for white. Returning -1 for an unreadable pixel makes the “brighter than” assertions fail. It would satisfy a “darker than” assertion, which is a gap listed below.

The five tests

Test	What it proves	Pixels read
testNoShadowLayerPixelsBeforeShadowIsAllowedToAppear	With shadowVisible off, the margin is pure white	three points 6 points above the bottom edge: left, centre, right
testShadowIsSoftAndNeverAHardEdgedBox	The shadow fades out before the window edge	bottom edge, bottom left corner, and a point in the bottom margin
testOpenShapeItselfIsSolidBlack	The open notch body is black	one point beside the tab bar
testControllerEnablesShadowOnlyAfterTheOpenAnimationAndDropsItInstantlyOnClose	State, not pixels: the flag follows the animation	none
testQuickOpenCloseNeverLeavesTheShadowOn	A flick across the notch leaves no shadow	none

The first one is the direct regression test for the black block. It renders the open shape with shadowVisible set to false and requires every sampled margin pixel to be brighter than 0.97.

for (x, y) in [(6.0, panelSize.height - 6), (panelSize.width / 2, panelSize.height - 6), (panelSize.width - 6, panelSize.height - 6)] {
    XCTAssertGreaterThan(luminance(rep, x: x, y: y), 0.97, "dark pixels at (\(x), \(y)) before the shadow is enabled")
}

If anyone brings back an always on shadow layer, those pixels go dark and the failure message names the point.

The second one checks the opposite: with the shadow on, it must be visible under the notch and gone at the window edge.

XCTAssertGreaterThan(nearBottomEdge, 0.93, "shadow must fade out before the window edge (hard cut-off)")
XCTAssertGreaterThan(corner, 0.97, "corner of the margin must be untouched")
XCTAssertLessThan(luminance(rep, x: panelSize.width / 2, y: belowShape), 0.99, "a shadow should be visible under the notch")

The thresholds are not copied from a reference. They are loose on purpose. 0.99 says “a little darker than white”, and 0.93 says “nearly white”. A tight threshold would fail on a one point change of blur radius, which is a design tweak and not a bug.

The wrong turn: a pixel that was an icon

testOpenShapeItselfIsSolidBlack asks one thing: is the body of the open notch black. The first version sampled the horizontal centre of the panel, 30 points above the bottom of the notch.

XCTAssertLessThan(luminance(rep, x: panelSize.width / 2, y: NotchModel.openSize.height - 30), 0.15)

That is the version in 969c58b. Later on the same day, 7 October 2026, the commit that fixed the test (7df40ae) gave its reason in the message: the sample at the centre of the tab bar had to be fixed now that there were ten tabs. The source comment in the fix gives the cause.

// Sample beside the tab bar, not its centre, where an icon sits when the tab count is odd.
XCTAssertLessThan(luminance(rep, x: m + 40, y: NotchModel.openSize.height - 40), 0.15)

The reasoning is geometric, and the numbers come from NotchRootView.swift. The relevant lines are these.

tabBar
    .padding(.top, 8)
    .padding(.bottom, 16)
// inside tabBar: HStack(spacing: 4) { ForEach(tabs) { ... .frame(height: 30) } }

In words: The tab bar is an HStack centred in the notch, 30 points tall, with 16 points of padding below it. In a notch that is 296 points tall, and assuming the content fills the shape, the bar occupies the vertical band from 250 to 280. The old sample was at y = 296 - 30 = 266, inside that band. It was also at the horizontal centre, which is exactly where a centred row puts its middle element when the number of elements is odd. A tab icon is drawn in a light colour, so the sample read a bright pixel and the test reported that the notch was not solid black. The notch was fine.

The failure reads as a number above 0.15 where the shape is black, and the notch looks right on screen. Nothing in it points at an icon. The cause only appears from knowing what the view draws at that coordinate.

The repair did not move the sample to another spot in the tab bar. It moved it out of the bar altogether: 40 points inside the left edge of the shape (m + 40, in a panel whose notch begins at m), and 40 points above the bottom. A tab bar of ten icons does not reach the left side. The assertion now only depends on the shape, which is the thing it is named after.

What was not verified for this post: the exact pixel position of the middle tab at the time of the failure. The test cannot be run without an Xcode build, so the claim rests on the commit message, the code comment and the layout arithmetic above. The case for an odd count is the code comment’s. The commit message mentions ten tabs, which is even, and the selected tab is wider than the others because it shows its title, so the centre of the bar can land on an icon for either count. The general lesson does not depend on which.

Numbers, with the method

All values come from reading the repository on 4 November 2026. No test was run for this post.

Fact	Source	Value
Test file length	wc -l Tests/NotchStarTests/ShadowArtifactTests.swift	91 lines
Number of tests in the file	grep -c "func test"	5
Open notch size	Tokens.Size.openNotch	660 by 296 points
Shadow margin	NotchModel.shadowMargin	30 points
Rendered panel	computed from the two lines above	720 by 326 points
Settle time before drawing	RunLoop.main.run(until:) in render	0.4 seconds
Open animation wait in the controller	Task.sleep(for: .milliseconds(400)) in NotchController	400 milliseconds
Shadow after the fix	.shadow(color: .black.opacity(0.45), radius: 12, y: 6)	radius 12, offset 6, opacity 0.45
Shadow before the fix	same line in the earlier commit	radius 20, offset 10, opacity 0.5

The fix changed two things in one commit: when the shadow appears, and how big it is. The smaller radius and offset keep the blur inside the 30 point margin, which is what the comment means by “stays well inside the transparent margin so it never clips”. The edge check guards that margin: a blur larger than it would be sliced by the window and show as a straight line.

What is still bad

• The render waits a fixed 0.4 seconds with RunLoop.main.run. That is a guess about how long SwiftUI needs. On a slow machine it could sample before layout settles, and nothing in the test would say so.
• The thresholds were chosen by judgement, not measured against a range of desktops. They are tuned for a white background. A dark desktop is not tested.
• Three of the five tests read pixels, and all of them are about the shadow or the shape. The tab bar, the header and the tool content are not covered by any pixel test, so a visual regression in them would not be caught by this file.
• luminance returns -1 for an unreadable pixel. That fails a “brighter than” check and passes a “darker than” check, so testOpenShapeItselfIsSolidBlack would pass on a failed read.
• The rendered output is from cacheDisplay, not from the screen. Real compositing, such as the panel’s window shadow and any display colour profile, is not part of it.
• The test builds an NSWindow, so it needs a Mac. It cannot run on a plain Linux CI machine.
• A pixel test says that a point is dark. It does not say why. The failure message has to carry the diagnosis, which is why each assertion has one.
• The NOTCH_RENDER_DIR environment variable writes the two renders to PNG for a human to look at. Looking at them is manual, and nothing compares them with a stored image.

Try it yourself

This works in any SwiftUI view. Start with a small view and one threshold.

let host = NSHostingView(rootView: MyView().background(Color.white))
host.frame = CGRect(x: 0, y: 0, width: 300, height: 200)
host.layoutSubtreeIfNeeded()
let rep = host.bitmapImageRepForCachingDisplay(in: host.bounds)!
host.cacheDisplay(in: host.bounds, to: rep)

The steps below build on that skeleton.

1. Render over a known background. Wrap the view in .background(Color.white), host it in an NSHostingView with an explicit frame, and call layoutSubtreeIfNeeded().
2. Draw it to a bitmap. Use bitmapImageRepForCachingDisplay(in:) and cacheDisplay(in:to:). You do not need to show a window.
3. Write a luminance helper. Scale view points to pixels with pixelsWide / frame.width, then read colorAt(x:y:) and apply 0.299, 0.587 and 0.114.
4. Pick points that only your subject can reach. Draw the layout on paper with real padding values. Avoid anything a control, text or an icon can cover.
5. Write thresholds as loose as the design allows. “Brighter than 0.97” detects a block, while “equal to 0.9832” detects nothing useful.
6. Put the diagnosis in the message. Say what should be true, and where.
7. Write the PNG when an environment variable is set. When a threshold fails, look at the picture before you change the number.
8. Run it once with the bug. Remove the fix, see the test fail for the right reason, and put the fix back.

Cheat sheet

• Render the real view at the real size, not a copy
• Use a flat known background so brightness equals the thing under test
• Scale points to pixels so the test is the same on 1x and 2x displays
• Return a failing value for an unreadable pixel
• Sample a point no control, label or icon can cover
• Keep thresholds loose enough that a design tweak does not break them
• Put the expected behaviour in the assertion message
• Pair pixel tests with state tests for timing
• Write a PNG on request, to look at the render
• Break the product once to see the test fail

Idea	Value used in ShadowArtifactTests
Margin pixels before the shadow is allowed	luminance above 0.97
Window edge after the shadow fades	luminance above 0.93
Visible shadow under the notch	luminance below 0.99
Solid black notch body	luminance below 0.15
Luma weights	0.299, 0.587, 0.114

Closing

This test belongs to NotchStar, a Mac app that puts ten tools in the notch. If you want to try it, it is at notchstar.com, with a 7 day free trial.