Tech blog by Tobias Fenster

Lessons learned for AI assistant usage with Docker Model Runner on ARM Windows

Sun, 01 Feb 2026 19:25:33 +0000

This is just a quick one to share two lessons I learned in the last few days when working with AI assistants and Docker Model Runner

Lesson 1: If you can’t connect, there might be an easy fix

Working with an ARM-based Windows laptop is fantastic, but it comes with a few drawbacks. I encountered one of these when I tried to follow a Docker Blog post about OpenCode with DMR. OpenCode is an open source AI coding agent and the blog post explains how you can connect it to DMR. As I have a Qualcomm Adreno GPU, I wanted to use that for inference, but here things got a bit tricky:

Windows (ARM) supports the Qualcomm 6xx series or later through the llama.cpp engine
Windows with WSL2 supports NVIDIA CUDA GPUs

This means that while I can technically run Docker Model Runner in WSL2, that setup can’t use my GPU. Unfortunately, OpenCode doesn’t support Windows ARM at the moment. That means that I have to use DMR running on the Windows “host” while OpenCode is running in WSL. Not a problem at first glance, because OpenCode connects to DMR via TCP (typically http://localhost:12434) anyway. But if you keep the networking defaults, then localhost in WSL os something different to localhost on the Windows host. You can often reach the same stuff from WSL by using host.docker.internal, but that doesn’t work in this case because DMR only listens on 127.0.0.1.

But when I adressed this to the Docker team¹, I was made aware of mirrored mode networking for WSL. If that is in place, the issue is no longer there, because to quote from the docs: “Connect to Windows servers from within Linux using the localhost address 127.0.0.1”. So now I can talk to localhost from WSL and get the same TCP connection as from the Windows host. That means that doing a curl to localhost as you see in the following screenshot hits DMR on the Windows side, hence giving me accelerated GPU inference.

The setup is explained well in the link mentioned above, but here is the configuration that works for me:

1
2
[wsl2]
networkingMode=mirrored

Lesson 2: Don’t just assume it works, but check

When I first tried to work with the GPU-based inference, I just assumed it works as I had looked up the documentation. However, that wasn’t true. docker model status revealed that llama.cpp failed to install:

1
2
3
Status:
llama.cpp: failed to install llama.cpp: Get "https://hub.docker.com/v2/namespaces/docker/repositories/docker-model-backend-llamacpp/tags/latest-opencl": dialing hub.docker.com:443 host via direct connection because Docker Desktop has no HTTPS proxy: connecting to hub.docker.com:443: dial tcp: lookup hub.docker.com: no such host
vllm: not installed

After setting DNS to 1.1.1.1 and 8.8.8.8 in my Docker Engine config like you see in the following snippet, the lookup worked:

1
2
3
4
5
6
7
8
{
  ...
  "dns": [
    "8.8.8.8",
    "1.1.1.1"
  ],
  ...
}

Restarting Docker Desktop then lead to the following docker model status output

1
2
3
4
5
6
docker model status
Docker Model Runner is running

Status:
llama.cpp: running llama.cpp latest-opencl (sha256:e12549787dbbb3b2c2e5d7a452edf7d64cd5a4a492679cc366b02f05d0efcc11) version: 091a46c
vllm: not installed

I also found the following entries in the ~\AppData\Local\Docker\log\host\inference-llama.cpp-server.log log:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
[2026-01-28T13:36:52.100268000Z] ggml_opencl: selected platform: 'QUALCOMM Snapdragon(TM)'
[2026-01-28T13:36:52.109192500Z] 
[2026-01-28T13:36:52.109192500Z] ggml_opencl: device: 'Qualcomm(R) Adreno(TM) X1-85 GPU (OpenCL 3.0 Qualcomm(R) Adreno(TM) X1-85 GPU)'
[2026-01-28T13:36:52.109192500Z] ggml_opencl: OpenCL driver: OpenCL 3.0 QUALCOMM build: 827.0 Compiler DX.18.05.00
[2026-01-28T13:36:52.109192500Z] ggml_opencl: vector subgroup broadcast support: true
[2026-01-28T13:36:52.109192500Z] ggml_opencl: device FP16 support: true
[2026-01-28T13:36:52.109192500Z] ggml_opencl: mem base addr align: 128
[2026-01-28T13:36:52.109192500Z] ggml_opencl: max mem alloc size: 2048 MB
[2026-01-28T13:36:52.109192500Z] ggml_opencl: device max workgroup size: 1024
[2026-01-28T13:36:52.109192500Z] ggml_opencl: SVM coarse grain buffer support: false
[2026-01-28T13:36:52.109192500Z] ggml_opencl: SVM fine grain buffer support: false
[2026-01-28T13:36:52.109788100Z] ggml_opencl: SVM fine grain system support: false
[2026-01-28T13:36:52.109788100Z] ggml_opencl: SVM atomics support: false
[2026-01-28T13:36:52.110290100Z] ggml_opencl: flattening quantized weights representation as struct of arrays (GGML_OPENCL_SOA_Q)
[2026-01-28T13:36:52.110290100Z] ggml_opencl: using kernels optimized for Adreno (GGML_OPENCL_USE_ADRENO_KERNELS)
[2026-01-28T13:37:12.093103100Z] ggml_opencl: loading OpenCL kernels.............................................................................
[2026-01-28T13:37:12.142341200Z] ggml_opencl: default device: 'Qualcomm(R) Adreno(TM) X1-85 GPU (OpenCL 3.0 Qualcomm(R) Adreno(TM) X1-85 GPU)'
[2026-01-28T13:37:12.153408400Z] main: setting n_parallel = 4 and kv_unified = true (add -kvu to disable this)
[2026-01-28T13:37:12.153408400Z] build: 1 (34ce48d) with Clang 19.1.7 for Windows ARM64
[2026-01-28T13:37:12.153408400Z] system info: n_threads = 6, n_threads_batch = 6, total_threads = 12
[2026-01-28T13:37:12.153408400Z] 
[2026-01-28T13:37:12.157120900Z] system_info: n_threads = 6 (n_threads_batch = 6) / 12 | CPU : NEON = 1 | ARM_FMA = 1 | LLAMAFILE = 1 | REPACK = 1 | 
[2026-01-28T13:37:12.157120900Z] 
[2026-01-28T13:37:12.157120900Z] init: using 11 threads for HTTP server
[2026-01-28T13:37:12.158494800Z] start: setting address family to AF_UNIX
[2026-01-28T13:37:12.165082800Z] main: loading model
[2026-01-28T13:37:12.166654200Z] srv    load_model: loading model '<HOME>\.docker\models\bundles\sha256\cad0bdb32dd3c9d0f915949d3434912eba79d66662ca16

Bottom line: Connecting to the “host” localhost from WSL is actually quite easy and for GPU-backed inference, make sure that it actually works…²

One of the, if not the biggest benefit of being a Docker Captain is that you can reach out the team directly and very often get pretty quick reactions. ↩
On a side note, the responses from OpenCode are still very slow. But that is a different topic where you can follow the progress at https://github.com/docker/model-runner/issues/606 and I might update this blog post with a third lesson (if it turns out there is one here as well) ↩

Logging and service monitoring in Windows Containers and AKS

Thu, 01 Jan 2026 09:44:26 +0000

To continue on the topic of running Windows Server 2025 containers and doing so on the Azure Kubernetes Service (AKS), this blog post explains how you can use LogMonitor and ServiceMonitor to run and monitor your Windows applications safely in AKS.

The TL;DR

To just give it a try, clone https://github.com/tfenster/logged-iis, which has all the sources you need. Then do the following:

Create a cluster using CreateCluster.ps1
Deploy my demo IIS container with kubectl apply -f iis.yaml and get the external IP with kubectl get services
Open the IIS starting page on that IP with something like http://9.141.62.204 (your IP will of course be different)
Check the logs in Azure Monitor, which should show you something like this

This is now an IIS container deployed to AKS that monitors the IIS Windows Service (W3SVC) through ServiceMonitor and puts the relevant logs to STDOUT through LogMonitor, which can then be collected in Azure Monitor. Now let’s take a look at the details how we got there.

The details: The container image with ServiceMonitor, LogMonitor and LogMonitorConfig.json

To get this to work from the container perspective, I used the same approach as explained previously to get IIS running in a Nano Server. What is special for the LogMonitor and ServiceMonitor, is the installation in the Dockerfile, based on a blog post about LogMonitor.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
FROM mcr.microsoft.com/windows/servercore:ltsc2025 AS download
WORKDIR /download
RUN powershell -Command \
    Invoke-WebRequest -Uri "https://github.com/microsoft/IIS.ServiceMonitor/releases/download/v2.0.1.10/ServiceMonitor.exe" -OutFile ".\ServiceMonitor.exe"; \
    Invoke-WebRequest -Uri "https://github.com/microsoft/windows-container-tools/releases/download/v2.1.3/LogMonitor.exe" -OutFile ".\LogMonitor.exe"

FROM mcr.microsoft.com/windows/nanoserver:ltsc2025
WORKDIR /install
COPY install_iis.cmd ./
COPY --from=download /download/ ./
USER ContainerAdministrator
RUN install_iis.cmd
WORKDIR /LogMonitor
COPY LogMonitorConfig.json ./
ENTRYPOINT ["C:\\install\\LogMonitor.exe", "C:\\install\\ServiceMonitor.exe", "W3SVC"]

In the end, I want to run in a Nano Server, but that base image doesn’t have any capability to download a file¹, so I went multi-stage with a Server Core base for the downloads first as you can see in lines 1-5. The files are then moved into the final stage in line 10. As mentioned above, the installation of IIS is the same as in my previous blog post. You can also see how LogMonitor is used with a nested ServiceMonitor call to monitor the IIS service in line 15. This runs LogMonitor with a configuration explained below, but also ServiceMonitor, which exits when the service state changes from SERVICE_RUNNING to anything else.

Potentially also interesting is the configuration of the collected logs in LogMonitorConfig.json, based on a sample in the official docs.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
{
    "LogConfig": {
        "sources": [
            {
                "type": "EventLog",
                "startAtOldestRecord": true,
                "eventFormatMultiLine": false,
                "channels": [
                    {
                        "name": "application",
                        "level": "error"
                    },
                    {
                        "name": "system",
                        "level": "error"
                    }
                ]
            },
            {
                "type": "File",
                "directory": "c:\\inetpub\\logs",
                "filter": "*.log",
                "includeSubdirectories": true
            },
            {
                "type": "ETW",
                "eventFormatMultiLine": false,
                "providers": [
                    {
                        "providerName": "IIS: WWW Server",
                        "providerGuid": "3A2A4E84-4C21-4981-AE10-3FDA0D9B0F83",
                        "level": "Information"
                    },
                    {
                        "providerName": "Microsoft-Windows-IIS-Logging",
                        "providerGuid": "7E8AD27F-B271-4EA2-A783-A47BDE29143B",
                        "level": "Information"
                    }
                ]
            }
        ]
    }
}

This collects information from the standard Windows event log (lines 4-18), from the IIS file-based logs (lines 19-24) and from the Event Tracing for Windows (ETW, lines 25-40). With that, we can see in STDOUT what goes on in the system generally and especially with IIS.

The details: AKS deployment

The AKS deployment in CreateCluster.ps1 is again very similar to what I shared before:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
az extension add --name aks-preview
az extension update --name aks-preview

az feature register --namespace "Microsoft.ContainerService" --name "AksWindows2025Preview"
az feature show --namespace Microsoft.ContainerService --name AksWindows2025Preview
az provider register --namespace Microsoft.ContainerService

$name = "akswin2025"
$region = "germanywestcentral"
$vmSize = "Standard_B4ls_v2"
az group create --name $name --location $region
az aks create -g $name -n $name --node-count 1 -s $vmSize --tier free --network-plugin azure --enable-app-routing --no-ssh-key --vm-set-type VirtualMachineScaleSets --network-plugin azure --enable-azure-monitor-metrics
az aks nodepool add --resource-group $name --cluster-name $name --os-type Windows --os-sku Windows2025 --name npwin --node-count 1 --ssh-access disabled --enable-fips-image
az aks enable-addons --addon monitoring --name $name --resource-group $name
az aks get-credentials --resource-group $name --name $name --overwrite-existing

Notable differences are:

The --enable-azure-monitor-metrics parameter in line 12 when the cluster is created. This enables monitoring generally for the AKS cluster via a Azure Monitor workspace. To learn more, check the official docs.
The monitoring addon, enabled via az aks enable-addons --addon monitoring... in line 14. This enables the collection of container logs. Again, to learn more, check the official docs

As explained above, LogMonitor with the right LogMonitorConfig.json makes sure that the right log entries end up in STDOUT and with the AKS setup explained here, that ends up in the workspace. With a KQL query like the following, also visible in the screenshot above, we can get e.g. the relevant parts from the IIS log files as they are also in STDOUT now:

1
2
3
4
5
6
7
8
9
ContainerLog
| union (ContainerLogV2
    | project-rename ContainerID = ContainerId
    | where ContainerName == "iis")
| extend d = parse_json(LogMessage. LogEntry)
| where LogMessage.Source == "File"
| extend Logline = d.Logline
| order by TimeGenerated desc
| project Logline

The deployment itself is trivial as you can see in iis.yaml. The only maybe relevant part is the usage of the tobiasfenster/logged-iis:nanoserver-ltsc2025 image, which is created using the Dockerfile above.

I hope this gives you an idea how to easily set up monitoring and logging for Windows containers in AKS, making them fit in the structure typically also used by Linux conatiners.

Of course, I could have used ADD in the Dockerfile, but that somehow isn’t intuitive for me, so I typically don’t. ↩

Running Windows Server 2025 workloads on Azure Kubernetes Service

Sun, 07 Dec 2025 11:56:43 +0000

A while ago, support for Windows Server 2025 in the Azure Kubernetes Service (AKS) was announced. Of course, I had to play around with it and in this blog post, I want to show you two scenarios: Running a Nano Server container enhanced by two FODs (see my recent blog post on the topic) and running a Microsoft Dynamics 365 Business Central container.

The TL;DR

To see it for yourself, you can do the following:

Clone my AKS Windows Server 2025 samples repo
Run CreateCluster.ps1 to create an AKS cluster with a Windows Server 2025 node pool
Run kubectl apply -f fod.yaml to deploy the nanoserver container with enabled IIS FOD
Run kubectl apply -f bc.yaml to deploy the Windows Server 2025 based BC container
Run kubectl get services to get the public IP addresses we need to use. This should give you something like

1
2
3
4
5
kubectl get services
NAME         TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)         AGE
bc           LoadBalancer   10.0.245.147   72.144.130.94   443:32055/TCP   15m
iis          LoadBalancer   10.0.36.200    9.141.62.204    80:30936/TCP    3h25m
kubernetes   ClusterIP      10.0.0.1       <none>          443/TCP         7d20h

Open http://<external IP of IIS>. In my example above, that would be http://9.141.62.204. This should give you the default IIS starting page

Open https://<external IP of BC>/BC?tenant=default. Again, in my example above, this would be https://72.144.130.94/BC?tenant=default. It uses a self-signed SSL certificate, so you will get a security warning. If you ignore that, you can log in with the user defined in the deployment, tfenster, and the defined password Super5ecret! to access the BC starting page

The details: Setting up the cluster

Let’s take a look at what CreateCluster.ps1 does:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
az extension add --name aks-preview
az extension update --name aks-preview

az feature register --namespace "Microsoft.ContainerService" --name "AksWindows2025Preview"
az feature show --namespace Microsoft.ContainerService --name AksWindows2025Preview
az provider register --namespace Microsoft.ContainerService

$name = "akswin2025"
$region = "germanywestcentral"
$vmSize = "Standard_B4ls_v2"
az group create --name $name --location $region
az aks create -g $name -n $name --node-count 1 -s $vmSize --tier free --network-plugin azure --enable-app-routing --no-ssh-key --vm-set-type VirtualMachineScaleSets --network-plugin azure
az aks nodepool add --resource-group $name --cluster-name $name --os-type Windows --os-sku Windows2025 --name npwin --node-count 1 --ssh-access disabled --enable-fips-image
az aks get-credentials --resource-group $name --name $name --overwrite-existing

First, since the feature we want is in preview, we add and potentially update the aks-preview extension to the Azure CLI (lines 1 and 2). Next, we register and show the AksWindows2025Preview feature and register the AKS provider (lines 4-6). Next, we set up the name, region and VM size for the cluster (lines 8-10). After that, we create the resource group and AKS cluster (lines 11 and 12), and then we add the Windows Server 2025 node pool (line 13). Finally, we get the credentials to interact with our cluster via kubectl (line 14). The entire script should produce an output similar to this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
PWSH C:\Users\tobia> az group create --name $name --location $region
{
  "id": "/subscriptions/94670b10-08d0-4d17-bcfe-e01f701be9ff/resourceGroups/akswin2025",
  "location": "germanywestcentral",
  "managedBy": null,
  "name": "akswin2025",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}
PWSH C:\Users\tobia>
PWSH C:\Users\tobia> az aks create -g $name -n $name --node-count 1 -s $vmSize --tier free --network-plugin azure --enable-app-routing --no-ssh-key --vm-set-type VirtualMachineScaleSets --network-plugin azure
Default SSH key behavior will change. When no SSH key parameters are provided, the command will behave as if '--no-ssh-key' was passed instead of failing in next breaking change release(2.80.0) scheduled for Nov 2025.
Argument '--enable-app-routing' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
The behavior of this command has been altered by the following extension: aks-preview
The new node pool will enable SSH access, recommended to use '--ssh-access disabled' option to disable SSH access for the node pool to make it more secure.
{
  "aadProfile": null,
  "addonProfiles": null,
  "agentPoolProfiles": [
    {
      "artifactStreamingProfile": null,
      "availabilityZones": null,
      "capacityReservationGroupId": null,
      "count": 1,
      "creationData": null,
      "currentOrchestratorVersion": "1.32.9",
      "eTag": "401ba2c0-78b0-4a5e-9679-3579eb7519f3",
      "enableAutoScaling": false,
      "enableEncryptionAtHost": false,
      "enableFips": false,
      "enableNodePublicIp": false,
      "enableUltraSsd": false,
      "gatewayProfile": null,
      "gpuInstanceProfile": null,
      "gpuProfile": null,
      "hostGroupId": null,
      "kubeletConfig": null,
      "kubeletDiskType": "OS",
      "linuxOsConfig": null,
      "localDnsProfile": null,
      "maxCount": null,
      "maxPods": 30,
      "messageOfTheDay": null,
      "minCount": null,
      "mode": "System",
      "name": "nodepool1",
      "networkProfile": {
        "allowedHostPorts": null,
        "applicationSecurityGroups": null,
        "nodePublicIpTags": null
      },
      "nodeCustomizationProfile": null,
      "nodeImageVersion": "AKSUbuntu-2204gen2containerd-202511.07.0",
      "nodeInitializationTaints": null,
      "nodeLabels": null,
      "nodePublicIpPrefixId": null,
      "nodeTaints": null,
      "orchestratorVersion": "1.32",
      "osDiskSizeGb": 128,
      "osDiskType": "Managed",
      "osSku": "Ubuntu",
      "osType": "Linux",
      "podIpAllocationMode": null,
      "podSubnetId": null,
      "powerState": {
        "code": "Running"
      },
      "provisioningState": "Succeeded",
      "proximityPlacementGroupId": null,
      "scaleDownMode": "Delete",
      "scaleSetEvictionPolicy": null,
      "scaleSetPriority": null,
      "securityProfile": {
        "enableSecureBoot": false,
        "enableVtpm": false,
        "sshAccess": "LocalUser"
      },
      "spotMaxPrice": null,
      "status": null,
      "tags": null,
      "type": "VirtualMachineScaleSets",
      "upgradeSettings": {
        "drainTimeoutInMinutes": null,
        "maxBlockedNodes": null,
        "maxSurge": "10%",
        "maxUnavailable": "0",
        "minSurge": null,
        "nodeSoakDurationInMinutes": null,
        "undrainableNodeBehavior": null
      },
      "upgradeSettingsBlueGreen": {
        "batchSoakDurationInMinutes": null,
        "drainBatchSize": null,
        "drainTimeoutInMinutes": null,
        "finalSoakDurationInMinutes": null
      },
      "upgradeStrategy": "Rolling",
      "virtualMachineNodesStatus": null,
      "virtualMachinesProfile": null,
      "vmSize": "Standard_B4ls_v2",
      "vnetSubnetId": null,
      "windowsProfile": null,
      "workloadRuntime": "OCIContainer"
    }
  ],
  "aiToolchainOperatorProfile": null,
  "apiServerAccessProfile": null,
  "autoScalerProfile": null,
  "autoUpgradeProfile": {
    "nodeOsUpgradeChannel": "NodeImage",
    "upgradeChannel": null
  },
  "azureMonitorProfile": null,
  "azurePortalFqdn": "akswin2025-akswin2025-94670b-vqa7xrf8.portal.hcp.germanywestcentral.azmk8s.io",
  "bootstrapProfile": {
    "artifactSource": "Direct",
    "containerRegistryId": null
  },
  "creationData": null,
  "currentKubernetesVersion": "1.32.9",
  "disableLocalAccounts": false,
  "diskEncryptionSetId": null,
  "dnsPrefix": "akswin2025-akswin2025-94670b",
  "eTag": "ced84e53-96a5-4e6c-b33c-d1b732d81f34",
  "enableNamespaceResources": null,
  "enableRbac": true,
  "extendedLocation": null,
  "fqdn": "akswin2025-akswin2025-94670b-vqa7xrf8.hcp.germanywestcentral.azmk8s.io",
  "fqdnSubdomain": null,
  "hostedSystemProfile": {
    "enabled": false
  },
  "httpProxyConfig": null,
  "id": "/subscriptions/94670b10-08d0-4d17-bcfe-e01f701be9ff/resourcegroups/akswin2025/providers/Microsoft.ContainerService/managedClusters/akswin2025",
  "identity": {
    "delegatedResources": null,
    "principalId": "4532f452-5dba-4965-9b9f-20d7d15ce9c5",
    "tenantId": "539f23a3-6819-457e-bd87-7835f4122217",
    "type": "SystemAssigned",
    "userAssignedIdentities": null
  },
  "identityProfile": {
    "kubeletidentity": {
      "clientId": "b37ac7a2-8cf1-4144-af8e-83d73649cb3f",
      "objectId": "44393190-32fe-406a-b7d9-801657bafc14",
      "resourceId": "/subscriptions/94670b10-08d0-4d17-bcfe-e01f701be9ff/resourcegroups/MC_akswin2025_akswin2025_germanywestcentral/providers/Microsoft.ManagedIdentity/userAssignedIdentities/akswin2025-agentpool"
    }
  },
  "ingressProfile": {
    "gatewayApi": null,
    "webAppRouting": {
      "defaultDomain": {
        "enabled": false
      },
      "dnsZoneResourceIds": null,
      "enabled": true,
      "identity": {
        "clientId": "633a3b19-828a-4281-8cd5-c4a29eb143e8",
        "objectId": "a31f7c1d-538c-46ff-a578-75c372f4c433",
        "resourceId": "/subscriptions/94670b10-08d0-4d17-bcfe-e01f701be9ff/resourcegroups/MC_akswin2025_akswin2025_germanywestcentral/providers/Microsoft.ManagedIdentity/userAssignedIdentities/webapprouting-akswin2025"
      },
      "nginx": {
        "defaultIngressControllerType": "AnnotationControlled"
      }
    }
  },
  "kind": "Base",
  "kubernetesVersion": "1.32",
  "linuxProfile": null,
  "location": "germanywestcentral",
  "maxAgentPools": 100,
  "metricsProfile": {
    "costAnalysis": {
      "enabled": false
    }
  },
  "name": "akswin2025",
  "networkProfile": {
    "advancedNetworking": null,
    "dnsServiceIp": "10.0.0.10",
    "ipFamilies": [
      "IPv4"
    ],
    "kubeProxyConfig": null,
    "loadBalancerProfile": {
      "allocatedOutboundPorts": null,
      "backendPoolType": "nodeIPConfiguration",
      "clusterServiceLoadBalancerHealthProbeMode": null,
      "effectiveOutboundIPs": [
        {
          "id": "/subscriptions/94670b10-08d0-4d17-bcfe-e01f701be9ff/resourceGroups/MC_akswin2025_akswin2025_germanywestcentral/providers/Microsoft.Network/publicIPAddresses/9d9a1620-981c-4cb0-aaef-4edcb9ad03ee",
          "resourceGroup": "MC_akswin2025_akswin2025_germanywestcentral"
        }
      ],
      "enableMultipleStandardLoadBalancers": null,
      "idleTimeoutInMinutes": null,
      "managedOutboundIPs": {
        "count": 1,
        "countIpv6": null
      },
      "outboundIPs": null,
      "outboundIpPrefixes": null
    },
    "loadBalancerSku": "standard",
    "natGatewayProfile": null,
    "networkDataplane": "azure",
    "networkMode": null,
    "networkPlugin": "azure",
    "networkPluginMode": null,
    "networkPolicy": "none",
    "outboundType": "loadBalancer",
    "podCidr": null,
    "podCidrs": null,
    "podLinkLocalAccess": "IMDS",
    "serviceCidr": "10.0.0.0/16",
    "serviceCidrs": [
      "10.0.0.0/16"
    ],
    "staticEgressGatewayProfile": null
  },
  "nodeProvisioningProfile": {
    "defaultNodePools": "Auto",
    "mode": "Manual"
  },
  "nodeResourceGroup": "MC_akswin2025_akswin2025_germanywestcentral",
  "nodeResourceGroupProfile": null,
  "oidcIssuerProfile": {
    "enabled": false,
    "issuerUrl": null
  },
  "podIdentityProfile": null,
  "powerState": {
    "code": "Running"
  },
  "privateFqdn": null,
  "privateLinkResources": null,
  "provisioningState": "Succeeded",
  "publicNetworkAccess": null,
  "resourceGroup": "akswin2025",
  "resourceUid": "692b12472e7971000183169c",
  "schedulerProfile": null,
  "securityProfile": {
    "azureKeyVaultKms": null,
    "customCaTrustCertificates": null,
    "defender": null,
    "imageCleaner": null,
    "imageIntegrity": null,
    "kubernetesResourceObjectEncryptionProfile": null,
    "nodeRestriction": null,
    "workloadIdentity": null
  },
  "serviceMeshProfile": null,
  "servicePrincipalProfile": {
    "clientId": "msi",
    "secret": null
  },
  "sku": {
    "name": "Base",
    "tier": "Free"
  },
  "status": null,
  "storageProfile": {
    "blobCsiDriver": null,
    "diskCsiDriver": {
      "enabled": true,
      "version": "v1"
    },
    "fileCsiDriver": {
      "enabled": true
    },
    "snapshotController": {
      "enabled": true
    }
  },
  "supportPlan": "KubernetesOfficial",
  "systemData": null,
  "tags": null,
  "type": "Microsoft.ContainerService/ManagedClusters",
  "upgradeSettings": null,
  "windowsProfile": {
    "adminPassword": null,
    "adminUsername": "azureuser",
    "enableCsiProxy": true,
    "gmsaProfile": null,
    "licenseType": null
  },
  "workloadAutoScalerProfile": {
    "keda": null,
    "verticalPodAutoscaler": null
  }
}
PWSH C:\Users\tobia> az aks nodepool add --resource-group $name --cluster-name $name --os-type Windows --os-sku Windows2025 --name npwin --node-count 1 --ssh-access disabled --enable-fips-image
Argument '--ssh-access' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
The behavior of this command has been altered by the following extension: aks-preview
{
  "artifactStreamingProfile": null,
  "availabilityZones": null,
  "capacityReservationGroupId": null,
  "count": 1,
  "creationData": null,
  "currentOrchestratorVersion": "1.32.9",
  "eTag": "745d5d2e-c47c-479d-8fdc-98962788f5ca",
  "enableAutoScaling": false,
  "enableEncryptionAtHost": false,
  "enableFips": true,
  "enableNodePublicIp": false,
  "enableUltraSsd": false,
  "gatewayProfile": null,
  "gpuInstanceProfile": null,
  "gpuProfile": null,
  "hostGroupId": null,
  "id": "/subscriptions/94670b10-08d0-4d17-bcfe-e01f701be9ff/resourcegroups/akswin2025/providers/Microsoft.ContainerService/managedClusters/akswin2025/agentPools/npwin",
  "kubeletConfig": null,
  "kubeletDiskType": "OS",
  "linuxOsConfig": null,
  "localDnsProfile": null,
  "maxCount": null,
  "maxPods": 30,
  "messageOfTheDay": null,
  "minCount": null,
  "mode": "User",
  "name": "npwin",
  "networkProfile": {
    "allowedHostPorts": null,
    "applicationSecurityGroups": null,
    "nodePublicIpTags": null
  },
  "nodeCustomizationProfile": null,
  "nodeImageVersion": "AKSWindows-2025-gen2-26100.6905.251027",
  "nodeInitializationTaints": null,
  "nodeLabels": null,
  "nodePublicIpPrefixId": null,
  "nodeTaints": null,
  "orchestratorVersion": "1.32",
  "osDiskSizeGb": 300,
  "osDiskType": "Ephemeral",
  "osSku": "Windows2025",
  "osType": "Windows",
  "podIpAllocationMode": null,
  "podSubnetId": null,
  "powerState": {
    "code": "Running"
  },
  "provisioningState": "Succeeded",
  "proximityPlacementGroupId": null,
  "resourceGroup": "akswin2025",
  "scaleDownMode": "Delete",
  "scaleSetEvictionPolicy": null,
  "scaleSetPriority": null,
  "securityProfile": {
    "enableSecureBoot": false,
    "enableVtpm": false,
    "sshAccess": "Disabled"
  },
  "spotMaxPrice": null,
  "status": null,
  "tags": null,
  "type": "Microsoft.ContainerService/managedClusters/agentPools",
  "typePropertiesType": "VirtualMachineScaleSets",
  "upgradeSettings": {
    "drainTimeoutInMinutes": null,
    "maxBlockedNodes": null,
    "maxSurge": "10%",
    "maxUnavailable": "0",
    "minSurge": null,
    "nodeSoakDurationInMinutes": null,
    "undrainableNodeBehavior": null
  },
  "upgradeSettingsBlueGreen": {
    "batchSoakDurationInMinutes": null,
    "drainBatchSize": null,
    "drainTimeoutInMinutes": null,
    "finalSoakDurationInMinutes": null
  },
  "upgradeStrategy": "Rolling",
  "virtualMachineNodesStatus": null,
  "virtualMachinesProfile": null,
  "vmSize": "Standard_D8d_v5",
  "vnetSubnetId": null,
  "windowsProfile": {
    "disableOutboundNat": null
  },
  "workloadRuntime": "OCIContainer"
}
PWSH C:\Users\tobia> az aks get-credentials --resource-group $name --name $name --overwrite-existing
The behavior of this command has been altered by the following extension: aks-preview
Merged "akswin2025" as current context in C:\Users\tobia\.kube\config

If you then use a tool like AKS Desktop (highly recommended), you should see your cluster including the Windows Server 2025 node:

The details: Creating and deploying the IIS (and SSH client) container image

Creating the container image works very much along the lines of what I showed in my first blog post about FODs: We use a Dockerfile that calls a script to install the FOD. The difference this time is that I also wanted to demonstrate installing a second FOD: the OpenSSH client. The installation process is very similar; basically only the name in line 1 is different, as seen in install_ssh_client.cmd:

1
2
3
4
5
6
DISM /Online /Add-Capability /CapabilityName:Microsoft.NanoServer.OpenSSH.Client /NoRestart
if errorlevel 3010 (
    echo The specified optional feature requested a reboot which was suppressed.
    exit /b 0
)
@echo off

The Dockerfile consequently also needs a reference to that script, as you can see in line 6

1
2
3
4
5
6
7
8
FROM mcr.microsoft.com/windows/nanoserver:ltsc2025
WORKDIR /install
COPY install_*.cmd ./
USER ContainerAdministrator
RUN install_iis.cmd
RUN install_ssh_client.cmd
USER ContainerUser
CMD ["ping", "-t", "localhost"]

A simple docker build -t tobiasfenster/fod:v0.0.6 . followed by a docker push tobiasfenster/fod:v0.0.6 creates and pushes that image. Now, let’s run this in AKS! Before we can do that, however, we need to consider how we want to use the SSH client. In my case, I need a private key to connect. First, I need to make the key available in the AKS cluster using this command:

1
kubectl create secret generic ssh-key-secret --from-file=id_rsa=c:/users/tobia/.ssh/id_azure

Of course, your SSH key most likely won’t be in c:\users\tobia\.ssh\id_azure, so you’ll need to adjust that. With that in place, we can kubectl apply -f .\fod.yaml with the fod.yaml looking like this

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
apiVersion: apps/v1
kind: Deployment
metadata:
  name: iis
  labels:
    app: iis
spec:
  replicas: 1
  template:
    metadata:
      name: iis
      labels:
        app: iis
    spec:
      nodeSelector:
        "kubernetes.io/os": windows
      containers:
      - name: iis
        image: tobiasfenster/fod:v0.0.6
        resources:
          limits:
            cpu: 1
            memory: 800M
        ports:
          - containerPort: 80
        volumeMounts:
        - name: ssh-key-volume
          mountPath: c:/ssh-key
          readOnly: true
      volumes:
      - name: ssh-key-volume
        secret:
          secretName: ssh-key-secret
  selector:
    matchLabels:
      app: iis
---
apiVersion: v1
kind: Service
metadata:
  name: iis
spec:
  type: LoadBalancer
  ports:
  - protocol: TCP
    port: 80
    name: http
  selector:
    app: iis

The first part (lines 1-36) is the deployment which containes the container. The most interesting lines are 18 and 19, which define the container; 15 and 16 which make sure that it ends up on a Windows node; and 26-30 which mount the SSH key into the container. Due to lines 38-49, the container id available via the external IP as explained in the TL;DR. Therefore, this is all that is needed for the IIS part.

Now what if we want to make an outgoing SSH connection? We have the SSH client installed through the FOD, and the private key is available through the secret and volume. However, we must properly define the permissions, otherwise SSH will decline to use the key, displaying an error message such as this:

1
2
3
4
5
6
7
Bad permissions. Try removing permissions for user: NT AUTHORITY\\Authenticated Users (S-1-5-11) on file c:/ssh-key/id_rsa.
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions for 'c:\\ssh-key\\id_rsa' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.

To fix that, you can run something like

1
2
3
4
5
6
copy c:\ssh-key\id_rsa .

icacls "c:\install\id_rsa" /inheritance:r
icacls "c:\install\id_rsa" /remove "NT AUTHORITY\Authenticated Users"
icacls "c:\install\id_rsa" /remove "BUILTIN\Users"
icacls "c:\install\id_rsa" /grant:r "%USERNAME%:R"

This copies the key to a new folder and fixes the permissions by breaking the inheritance and removing everyone before adding the current user. After that, you should be able to use the key to SSH into a remote machine.

You have now seen how to create and use a Windows Server 2025 Nano Server image to run a container in AKS that serves a webpage through IIS and establishes SSH connections. You might now be wondering which other FODs are available. It’s surprisingly difficult to find out, but you can run a Windows Server 2025 container and query for it like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
docker run --user ContainerAdministrator -ti mcr.microsoft.com/windows/nanoserver:ltsc2025
Microsoft Windows [Version 10.0.26100.7171]
(c) Microsoft Corporation. All rights reserved.

C:\>DISM /Online /Get-Capabilities /Format:Table

Deployment Image Servicing and Management tool
Version: 10.0.26100.5074

Image Version: 10.0.26100.7171

Capability listing:


---------------------------------------------------- | -----------
Capability Identity                                  | State
---------------------------------------------------- | -----------
Microsoft.NanoServer.ADSI~~~~                        | Not Present
Microsoft.NanoServer.BITS.Admin~~~~                  | Not Present
Microsoft.NanoServer.CommandLine.Utilities~~~~       | Not Present
Microsoft.NanoServer.Datacenter.WOWSupport~~~~       | Not Present
Microsoft.NanoServer.Globalization~~~~               | Not Present
Microsoft.NanoServer.IIS~~~~                         | Not Present
Microsoft.NanoServer.OpenSSH.Client~~~~              | Not Present
Microsoft.NanoServer.OpenSSH.Server~~~~              | Not Present
Microsoft.NanoServer.Performance.Monitoring~~~~      | Not Present
Microsoft.NanoServer.PowerShell.Cmdlets~~~~          | Not Present
Microsoft.NanoServer.RemoteFS.Client~~~~             | Not Present
Microsoft.NanoServer.StateRepository~~~~             | Not Present
Microsoft.NanoServer.WinMgmt.RuntimeDependencies~~~~ | Not Present
Microsoft.NanoServer.WinMgmt~~~~                     | Not Present

The operation completed successfully.

I wouldn’t be able to tell you exactly what those FODs do.

The details: Creating and deploying the BC container image

The second scenario is a Business Central container based on a Windows Server 2025 base image. Since these images are not readily available, you first need the bccontainerhelper PowerShell module. Then, you can run something like this to create the image:

1
New-BcImage -artifactUrl (Get-BCArtifactUrl -type sandbox -country w1 -accept_insiderEula -select NextMajor) -imageName tobiasfenster/mybc -baseImage "mcr.microsoft.com/businesscentral:ltsc2025"

In my case, this gave me the image tobiasfenster/mybc:sandbox-28.0.43121.0-w1-mt. Because of that, we can run a simple deployment to AKS like you see in bc.yaml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
apiVersion: apps/v1
kind: Deployment
metadata:
  name: bc
  labels:
    app: bc
spec:
  replicas: 1
  template:
    metadata:
      name: bc
      labels:
        app: bc
    spec:
      nodeSelector:
        "kubernetes.io/os": windows
      containers:
      - name: bc
        image: tobiasfenster/mybc:sandbox-28.0.43121.0-w1-mt
        ports:
        - containerPort: 443
        env:
        - name: username
          value: tfenster
        - name: password
          value: Super5ecret!
        - name: auth
          value: NavUserPassword
        - name: accept_eula
          value: "y"
        - name: accept_outdated
          value: "y"
  selector:
    matchLabels:
      app: bc
---
apiVersion: v1
kind: Service
metadata:
  name: bc
spec:
  type: LoadBalancer
  ports:
  - protocol: TCP
    port: 443
    name: https
  selector:
    app: bc

As in the previous example, the container is shown in lines 18 and 19, the Windows node selection is shown in lines 15 and 16, and the service that enables incoming traffic is shown in lines 37–48. The BC container has a special username and password setup (lines 23–28) and requires acceptance of the EULA and outdated image (lines 29–32) to run.

Of course, for real dev/test scenarios, you would need much more, but this gives you the basics to see that BC on Windows Server 2025 runs in AKS.

Using the Business Central MCP Server with Visual Studio Code or cagent

Wed, 05 Nov 2025 23:18:36 +0000

At the Directions EMEA 2025 conference, Microsoft announced the public preview of their Business Central MCP Server. In case you haven’t heard, MCP is an open-source standard to connect AI applications to external systems. In the case of the BC MCP Server, this means that AI applications can access Business Central. I don’t want to go into the details on the BC side, as I’m sure that Microsoft will do a good job in documenting that. But for very understandable reasons, Microsoft only supports the Microsoft Copilot Studio as client, even though you may still want to use it with other clients, such as Visual Studio Code with Github Copilot or cagent.

The TL;DR

Here’s what you need to do for that:

Get the code from the BcMCPProxy sample in the BCTech Github repo
Compile it into an executable file
Follow the instructions in the repository to create an app registration, which you will need to authenticate against Business Central
Configure the proxy as an MCP Server in your client
Start using it!

For example, this is what it looks like in VS Code:

The details: Getting the BcMCPProxy executable

The BC MCP Proxy is a .NET 8 application, which means that you need a .NET SDK to compile it. To make that easier, I have created a pull request that adds devcontainer support as well as build tasks for VS Code to make building self-contained executables easier. Until that PR is merged (if at all), you can clone my fork and open the samples/BcMCPProxy folder in a dev container. Once it has started, you can run the build tasks and select your operating system. In my case, this is Windows-ARM64, but of course your setup may differ. You will then have an executable file that you can copy to your host machine and use it in the MCP setup later.

Update: Thanks to a very fast review and merge by Microsoft, this is now part of Microsoft’s repo, so you can use that functionality directly without a need to go through my fork!

Configuring VS Code

One way to configure an MCP Server in VS Code is to use a .vscode/mcp.json file (see the official docs for all options). As explained in the repo mentioned above for Claude Desktop, you need to add some configuration options. It should look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
{
    "servers": {
        "bc": {
            "type": "stdio",
            "command": "C:\\Users\\tobia\\deleteme\\BcMCPProxy.exe",
            "args": [
                "--TenantId",
                "<tenant-id>",
                "--ClientId",
                "<client-id>",
                "--Environment",
                "<environment>",
                "--Company",
                "<company>",
                "--ConfigurationName",
                "<configuration-name>"
            ]
        }
    }
}

As you can see, you need to fill in the following:

the ID of your Entra ID tenant in line 8;
the client ID of your app registration in line 10;
the name of your BC environment where you have the MCP feature enabled and configured in line 12;
the name of the BC company you want to use in line 14;
the name of the MCP configuration in line 16. You can then click on “Start” and use the tools in GitHub Copilot Chat within VS Code, as shown in the video above!

Configuring cagent

Another client that works well is Docker’s cagent. In this case, the configuration is very similar, we only need to also add the LLM that we want to use. An example could look like this, using an Azure AI Foundry model:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
version: "1"

agents:
  root:
    description: An agent that interacts with Microsoft Dynamics 365 Business Central
    instruction: |
      You are an agent helping the user to interact with the ERP system Microsoft Dynamics 365 Business Central.
    model: cloud-gpt-5
    toolsets:
      - type: mcp
        command: C:\Users\tobia\deleteme\BcMCPProxy.exe
        args: [
            "--TenantId",
            "<tenant-id>",
            "--ClientId",
            "<client-id>",
            "--Environment",
            "<environment>",
            "--Company",
            "<company>",
            "--ConfigurationName",
            "<configuration-name>"
        ]

models:
  cloud-gpt-5:
    provider: azure
    model: gpt-5
    base_url: <your-azure-ai-foundry-endpoint>
    provider_opts:
      azure_api_version: 2025-01-01-preview

You can see the same setup as for VS Code in lines 10–23, as well as the model setup in lines 25–31 and the reference to it in line 8. With this configuration, the BC MCP Server can be used in a similar way:

With that, you should also be able to adapt the configuration for other clients. Have fun building applications and solving problems with it!

Debugging container image creation with a Dockerfile

Sun, 19 Oct 2025 18:25:24 +0000

If you have ever worked on even mildly complex container images based on Dockerfiles, then I am pretty sure that you know the pain of debugging when things go wrong. Typically, that meant juggling comments or removing parts of the Dockerfile to get an image during the intended build. Then, you would create a container and look at the resulting file system or run commands. Fortunately that has now changed as Docker has released a new feature that is available with the latest Docker Desktop and the latest VS Code Docker extension: Real debugging of the build process in VS Code! I’ll walk you through the main elements, but I would certainly also recommend reading the official announcement.

The TL;DR

Let’s assume that you have a Dockerfile that describes your container image. Then you need a launch configuration like this

1
2
3
4
5
6
7
{
    "name": "Docker: Build",
    "type": "dockerfile",
    "request": "launch",
    "dockerfile": "Dockerfile",
    "contextPath": "${workspaceFolder}",
}

With that in place, you can run the launch configuration as you would any other debug configuration by starting a debug session. If you have breakpoints in the Dockerfile, the debugger will stop, allowing you to view arguments and the file system, as well as access additional features, as explained below.

Pretty cool, right?

The details: My example and basic setup

As an example, I have created a simple .NET application using dotnet new console -n App. For that, I also created a Dockerfile, but to make things a bit more interesting, it is a multi-stage one:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0 AS build
ARG configuration=Release
WORKDIR /src
COPY ["App.csproj", "./"]
RUN dotnet restore "App.csproj"
COPY . .
WORKDIR "/src/."
RUN dotnet build "App.csproj" -c $configuration -o /app/build

FROM build AS publish
ARG configuration=Release
RUN dotnet publish "App.csproj" -c $configuration -o /app/publish /p:UseAppHost=false

FROM mcr.microsoft.com/dotnet/runtime:10.0 AS final
USER app
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "App.dll"]

You can see the three stages, build, publish and final. As the names say, build and publish are for building and publishing, both using the full .NET SDK base image and the last one is for running only and uses the runtime image.

As mentioned in the TL;DR above, you need a launch configuration for debugging, which I showed in the previous paragraph.

One thing I struggled with initially was that at least for me, it didn’t work directly on Windows. Instead, I had to open a WSL terminal and start VS Code from there.

The details: Debugging

Another lesson I learned immediately was that breakpoints don’t get hit on every command. By chance I first tried with line 2 with the ARG call. The debugger didn’t stop. Thinking it might be due to the multi-stage nature of the image, so I tried line 15 with the USER call in the final stage. once again, the debugger didn’t stop. Only when I set the breakpoints on lines 3 and 16 which have the WORKDIR command, did the debugger stop. Also if I set the breakpoint on the very first line and went step by step using F10, the ARG and USER lines were never hit.

But once you know that, it works very conveniently. E.g. take the very first line FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0 AS build. If you want to figure out the value of $BUILDPLATFORM, just stop on that line and you can check the value under VARIABLES > Arguments > platform on the right

A few steps later, after the /src folder has been created and the App.csproj file is copied in, you can see the folder structure and file appear under VARIABLES > File Explorer and you can even see the content of a (text) file under date and as a mouseover.

In my opinion, the most useful feature is the ability to access an interactive console in the container during the build process. You can access this feature by typing exec into the command bar of the debug console in VS Code. With this feature, you can interact with the container during the build process. For example, you can run commands like this:

You can see that a new file was generated and that I could make changes to the container’s file system. However, once I step to the next line, this happens:

As we now have an obj folder, you can see that the dotnet restore has happened. At the same time, the test.txt file that was created interactively has disappeared because those changes are revoked once we go to the next line in the debugger.

The details: Multi-stage images and debugging immediately

One thing that worked quite intuitively for me was using multi-stage images. Setting "stopOnEntry": true in your launch config causes the debugger to stop immediately upon entering the build process. Interestingly, though, it doesn’t stop on line 1, as you might imagine, but rather on the first line of the final stage.

If you use F10 to step over lines, you will never reach the other stages. However, if you use Step Into or hit F11 on line 17, which references another stage, the debugger will go to that stage. As a developer, I find this very intuitive because it’s the same way I would step into a function.

I hit a few early hiccups when trying out the new debugger, but once I understood how it worked and set it up correctly, it worked really well. I am quite sure that it will come in handy if I run into issues with my Dockerfiles!

Orchestrate multiple AI agents with cagent by Docker to create a BC/AL coding assistant

Sun, 07 Sep 2025 20:18:12 +0000

One of the easiest ways to lure me into trying something is labelling it as “experimental”. I absolutely love playing with early-stage tools, seeing the raw and basic ideas of the creators and maybe even help to shape them into something useful. So when I recently was made aware of a new tool created and shared by Docker called “cagent” in very early experimental stage, I of course couldn’t resist to give it a try.

The TL;DR

cagent is a multi-agent runtime that orchestrates AI agents with specialized capabilities and tools, and the interactions between agents. The example scenario I want to share with you is a Microsoft Dynamics 365 Business Central (programming language AL) coding assistant that orchestrates three agents: The first one develops the code based on a requirement, the second one reviews the code and the third one does the git handling (create a branch, checkout, commit). To allow it to work, the coding and review agents have access to the filesystem and shell to make sure they can see existing code which I want to extend. They also have access to the AL MCP Server by Stefan Maron to help them understand the existing code structure in Business Central. The git handling agent has access to the reference git MCP server.

With that I can give my multi-agent system in cagent the following task, based on an already existing application where one specific small feature is missing: “i want to add a flag on the item card to show whether an item is an alloy item or not. Note that I already have that on the item list, but I also want it on the item card. make the required changes to the already existing code in the app folder”. Here is how it does:

To be honest, it took me quite a few attempts to get there, but I really like the result. It isn’t perfect and it doesn’t generate perfect code. But it is a helpful starting point and I will test with more complex examples to see where the boundaries are (admittedly, this is basically the simplest possible example). Also note that I am using a coding assistant as an example, but this is not limited to coding. It could be agents for everything you can think of and the examples in the GitHub repo of cagent give you an idea of that.

Is this the only or even the best way to create and maintain multi-agent systems? I can’t claim to know enough about that landscape to really answer that. But as a developer, I love the simplicity, the configurability via YAML (which also makes it perfect for version control and storing it next to my source code) and the terminal interface. There are certainly fancier and more end-user friendly tools, but it absolutely hits a sweet spot for me.

The details: Defining the agents and their orchestration

To learn how cagent works, I would point you at the official docs, the readme and the usage file. The concept is really easy to understand and the YAML structure to define everything limited to the required elements. So I want make this a general introduction but point you at some key aspects:

First, we have a “root” agent, which gets the initial input and orchestrates the other agents. The root agent has access to subagents:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
agents:
  root:
    model: cloud-gpt-5-chat
    description: "Microsoft Dynamics 365 Business Central development assistant"
    instruction: |
      You are the leader of a team and sub-teams of AI Agents.
      Your task is to coordinate the team to complete the user's request.

      Here are the members in your team:
      <team_members>
      - developer_agent: Analyzes the code base and develops new code
      - reviewer_agent: Analyzes the changes done by the developer_agent and potentially suggests optimizations
      - git_agent: Can make git calls to create branches and commit changes
      </team_members>

      <WORKFLOW>
        1. Call the `developer_agent` to analyze the code base and propose new code
        2. Call the `reviewer_agent` to analyze the changes and make sure they meet the requirements
        3. Call the `git_agent` to create a branch and commit the changes
      </WORKFLOW>
      ...
    sub_agents:
      - developer_agent
      - reviewer_agent
      - git_agent

The agent needs to know which model to use (more on that later) and gets a basic description and instructions. Here we let it know that it has multiple agents at its disposal and how and in which order we want it to interact with them.

The other agents also have access to tools, as mentioned above. E.g., the developer_agent looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
  developer_agent:
    model: cloud-gpt-5-chat
    description: "Expert code analysis and development assistant"
    instruction: |
      ...
    toolsets:
      - type: shell
      - type: filesystem
      - type: todo
      - type: mcp
        command: npx
        args: ["al-mcp-server"]

Here you can see that it has access to the shell to execute commands, to the filesystem to read and write files and to todos to create a multi-step action. Those are built-in tools, so the name is enough. The last tool is the al-mcp-server which allows the agent to understand the AL code in compiled dependencies already in the project (“AL packages”) and work with them. In this case, the MCP server is called via npx. But we can also use the Docker MCP Gateway, which you can see in the configuration of the git_agent:

1
2
3
4
5
6
7
8
9
10
  git_agent:
    model: cloud-gpt-5-chat
    description: "Git expert who uses git to fullfil tasks"
    instruction: |
      ...
    toolsets:
      - type: shell
      - type: filesystem
      - type: mcp
        ref: docker:git

Here we only need the ref: docker:git to point it in the right direction.

As already briefly mentioned above, every agent also needs a model that it uses. And you already saw lines like this model: cloud-gpt-5-chat. The models are configured or made available in a separate section:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
models:
  cloud-gpt-4o:
    provider: openai
    model: gpt-4o
    base_url: https://openai-testtf.openai.azure.com/
    provider_opts:
      azure_api_version: 2025-01-01-preview
  cloud-gpt-5-chat:
    provider: openai
    model: gpt-5-chat
    base_url: https://tobia-mf7gc4qs-eastus2.cognitiveservices.azure.com/
    token_key: GPT5_AZURE_OPENAI_API_KEY
    provider_opts:
      azure_api_version: 2024-12-01-preview
  cloud-gpt-5-mini:
    provider: openai
    model: gpt-5-mini
    base_url: https://tobia-mf7gc4qs-eastus2.cognitiveservices.azure.com/
    token_key: GPT5_AZURE_OPENAI_API_KEY
    provider_opts:
      azure_api_version: 2024-12-01-preview

Here you can see three model definitions: cloud-gpt-4o, cloud-gpt-5-chat and cloud-gpt-5-mini. You probably saw it above, I only used cloud-gpt-5-chat as it got me by far the best results. Each of them needs a provider (in my case openai, but could also be anthropic, google or dmr for the Docker Model Runner) and a model, which is the name of the model to be used as defined by the provider like gpt-4o or gpt-5-chat. That would be enough for “regular” openai, google or anthropic as it now has all information and reads the corresponding API keys from environment variables like OPENAI_API_KEY. I however wanted to use Azure OpenAI. That requires a bit more configuration: At least the base_url where my Azure Open AI / Azure AI Foundry deployment is available and the azure_api_version to identify the version to be used. In this case, the API key is read by default from AZURE_OPENAI_API_KEY, but as you can see, I have different deployments for GPT-4o and GPT-5, so I had to use different keys. That is also configurable via the token_key, which is the name of the environment variable to read the key from.

The details: Support for Azure OpenAI / Azure AI Foundry and devcontainers

What was most impressive for me was the collaboration with the development team. I asked about Azure OpenAI support and it wasn’t there initially, but only a couple hours after I raised the request, there already was a PR. We discussed the implementation a bit, and the team even picked up a suggestion. I am confident that this will make it into one of the next releases, but, like me, you can easily build it yourself thanks to a multi-arch Dockerfile and build task. And I contributed devcontainer support, so setting up your own development environment is also very easy.

The details: Things to note during the coding session

Now if we take a closer look at the coding session, I want to point out a few things related to the topics above:

Here, you can see the root agent reaching out to the developer agent and telling it what to do. I find it very valuable to see how the root agent interpreted my input above, and how it passed it on to the developer agent. This is especially useful for debugging and understanding what happens in cases where the agents don’t work as expected.

In the second screenshot, you can see how the developer agent searches for information about code in the Business Central base application. However, because that code is not available as source code in this specific workspace, but only as a compiled package, it can’t find it. Now, the AL MCP Server kicks in:

As it is an additional tool and the first tool call in this session, cagent asks for permission to use the AL MCP Server, specifically the tool to discover all the compiled packages. I answer this with All, so no further permission requests pop up. After discovery, the agent can then call the tool to search for the specific page, “Item Card”, mentioned in my requirements. I like how cagent shows exactly how it calls the tool, making it easy for me to understand what is happening and potentially improve my prompt if it isn’t what I want.

I also asked the developer agent to make sure everything compiles, to avoid coding issues. This also works through a tool, this time the built-in shell. Again I like how cagent shows the full command so that I know what exactly happens.

With that, the developer agent is finished and the root agent hands it off to the reviewer agent. This is once again pointed out clearly by cagent:

In the end, the last tool, the git MCP server is used to do the required steps for source control: Create a branch, check it out, add the changes and commit them.

Overall, I like how cagent makes it very clear what happens, who calls whom, which tools are used and how. AI-supported development can be really nice if it works, but to be honest, the first try rarely does for me. Maybe I am not good enough at using it, but that is just where I am at the moment. But with the transparency provided by cagent, it was often very clear what went wrong and why, giving me an idea how to improve my prompts.

The details: The full cagent definition

To close it off, here is the full definition that I used. A lot of it is based on the code.yaml and writer.yaml examples in the cagent repo.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
version: "2"

agents:
  root:
    model: cloud-gpt-5-chat
    description: "Microsoft Dynamics 365 Business Central development assistant"
    instruction: |
      You are the leader of a team and sub-teams of AI Agents.
      Your task is to coordinate the team to complete the user's request.

      Here are the members in your team:
      <team_members>
      - developer_agent: Analyzes the code base and develops new code
      - reviewer_agent: Analyzes the changes done by the developer_agent and potentially suggests optimizations
      - git_agent: Can make git calls to create branches and commit changes
      </team_members>

      <WORKFLOW>
        1. Call the `developer_agent` to analyze the code base and propose new code
        2. Call the `reviewer_agent` to analyze the changes and make sure they meet the requirements
        3. Call the `git_agent` to create a branch and commit the changes
      </WORKFLOW>

      - Use the transfer_to_agent tool to call the right agent at the right time to answer the users question.
      - DO NOT transfer to multiple members at once
      - ONLY CALL ONE AGENT AT A TIME
      - When using the `transfer_to_agent` tool,  make exactly one call and wait for the result before making another. Do not batch or parallelize tool calls.

      General Guidelines:
      - Always analyze the user query to identify relevant metadata.
      - Use the most specific filter(s) possible to narrow down results.
      - Use tables to display data
      - Always include sources
    sub_agents:
      - developer_agent
      - reviewer_agent
      - git_agent

  developer_agent:
    model: cloud-gpt-5-chat
    description: "Expert code analysis and development assistant"
    instruction: |
      You are an expert developer with deep knowledge of code analysis, modification, and validation.

      Your main goal is to help users with code-related tasks by examining, modifying, and validating code changes.
      Always use conversation context/state or tools to get information. Prefer tools over your own internal knowledge.

      <TASK>
          # **Workflow:**

          # 1. **Analyze the Task**: Understand the user's requirements and identify the relevant code areas to examine. Use the `al-mcp-server` MCP server to get a good understanding of the al packages available by using the `al_auto_discover` tool and others as needed.

          # 2. **Code Examination**: 
          #    - Search for relevant code files and functions
          #    - Analyze code structure and dependencies
          #    - Identify potential areas for modification

          # 3. **Code Modification**:
          #    - Make necessary code changes
          #    - Ensure changes follow best practices
          #    - Maintain code style consistency

          # 4. **Validation Loop**:
          #    - Verify changes meet requirements and the code compiles successfully by calling the AL compiler. 
          #    - If issues found, return to step 3
          #    - Continue until all requirements are met

          # 5. **Documentation**:
          #    - Document significant changes
          #    - Update relevant comments
          #    - Note any important considerations
      </TASK>

      **Tools:**
      You have access to the following tools to assist you:

      * Filesystem tools for reading and writing code files
      * Search tools for finding relevant code
      * Shell access for running linters and validators
      * DuckDuckGo for research when needed
      * if you need it, the AL compiler is called alc.exe. you can find it in ~\.vscode-insiders\extensions\ms-dynamics-smb.al-17.0.1750311\bin\win32. make sure to specifiy the package cache path .alpackages when calling it

      **Constraints:**

      * **Never mention "tool_code", "tool_outputs", or "print statements" to the user.** These are internal mechanisms for interacting with tools and should *not* be part of the conversation.
      * Be thorough in code examination before making changes
      * Always validate changes before considering the task complete
      * Follow best practices and maintain code quality
      * Be proactive in identifying potential issues
      * Only ask for clarification if necessary, try your best to use all the tools to get the info you need
      * Don't show the code that you generated

    toolsets:
      - type: shell
      - type: filesystem
      - type: todo
      - type: mcp
        command: npx
        args: ["al-mcp-server"]

  reviewer_agent:
    model: cloud-gpt-5-chat
    description: "Expert code analysis and development assistant who reviews the results of others"
    instruction: |
      You are an expert developer with deep knowledge of code analysis, modification, and validation.

      Your main goal is to review the code changes in a specific folder. Use git to understand what is locally changes. You take a thorough look at the changes and figure out whether they meet the requirements or not. You propose optimizations if necessary.

      <TASK>
          # **Workflow:**

          # 1. **Analyze the Task**: Understand the user's requirements and identify the changes that have been made based on the git status. Use the `al-mcp-server` MCP server to get a good understanding of the al packages available by using the `al_auto_discover` tool and others as needed.

          # 2. **Code Examination**: 
          #    - Search for relevant code files and functions
          #    - Analyze code structure and dependencies
          #    - Identify potential areas for modification

          # 3. **Code Review**:
          #    - Check whether the chances truely meet the requirements
          #    - If yes, confirm that to the user
          #    - If no, suggest optimizations

          # 4. **Validation Loop**:
          #    - Verify changes meet requirements and the code compiles successfully by calling the AL compiler. 
          #    - If issues found, return to step 3
          #    - Continue until all requirements are met

          # 5. **Documentation**:
          #    - Document significant changes
          #    - Update relevant comments
          #    - Note any important considerations
      </TASK>

      **Tools:**
      You have access to the following tools to assist you:

      * Filesystem tools for reading and writing code files
      * Search tools for finding relevant code
      * Shell access for running linters and validators
      * DuckDuckGo for research when needed
      * If needed, the AL compiler is called alc.exe. you can find it in ~\.vscode-insiders\extensions\ms-dynamics-smb.al-17.0.1750311\bin\win32. make sure to specifiy the package cache path .alpackages when calling it

      **Constraints:**

      * **Never mention "tool_code", "tool_outputs", or "print statements" to the user.** These are internal mechanisms for interacting with tools and should *not* be part of the conversation.
      * Be thorough in code examination before making changes
      * Always validate changes before considering the task complete
      * Follow best practices and maintain code quality
      * Be proactive in identifying potential issues
      * Only ask for clarification if necessary, try your best to use all the tools to get the info you need
      * Don't show the code that you generated

    toolsets:
      - type: shell
      - type: filesystem
      - type: todo
      - type: mcp
        ref: docker:git
      - type: mcp
        command: npx
        args: ["al-mcp-server"]

  git_agent:
    model: cloud-gpt-5-chat
    description: "Git expert who uses git to fullfil tasks"
    instruction: |
      You are an expert git user that can check changes, create branches and commit and push changes.

      Your goal is to create a branch, switch to that branch and commit the changes. Don't push the changes, the user has to do that.

      **Tools:**
      You have access to the following tools to assist you:

      * Filesystem tools for reading and writing code files
      * Shell access for running linters and validators
      * The git MCP server to execute git commands

      **Constraints:**

      * **Never mention "tool_code", "tool_outputs", or "print statements" to the user.** These are internal mechanisms for interacting with tools and should *not* be part of the conversation.

    toolsets:
      - type: shell
      - type: filesystem
      - type: mcp
        ref: docker:git

models:
  cloud-gpt-4o:
    provider: openai
    model: gpt-4o
    base_url: https://openai-testtf.openai.azure.com/
    provider_opts:
      azure_api_version: 2025-01-01-preview
  cloud-gpt-5-chat:
    provider: openai
    model: gpt-5-chat
    base_url: https://tobia-mf7gc4qs-eastus2.cognitiveservices.azure.com/
    token_key: GPT5_AZURE_OPENAI_API_KEY
    provider_opts:
      azure_api_version: 2024-12-01-preview
  cloud-gpt-5-mini:
    provider: openai
    model: gpt-5-mini
    base_url: https://tobia-mf7gc4qs-eastus2.cognitiveservices.azure.com/
    token_key: GPT5_AZURE_OPENAI_API_KEY
    provider_opts:
      azure_api_version: 2024-12-01-preview

Azure DevOps can do that? Webhooks for advanced customizations

Fri, 01 Aug 2025 04:26:22 +0000

If you use Azure DevOps, you may have wanted for some kind of customization that wasn’t available out of the box. E.g. we want the area path of a work item to automatically change once it has reached a certain column in a board¹, but there is no feature that can do that. However, Azure DevOps does have Webhooks that we could use to achieve this. Setting up such a webhook is well-documented and quite easy, so I won’t get into the details here. But how does a backend need to look like to respond to that call? This I’ll explain thath in this blog post.

The TL;DR

If you just want to give it a try using a GitHub Codespace, fork my repository https://github.com/tfenster/azdevops-webhook-explorer and start the codespace. Everything should be preconfigured through the devcontainer.json configuration file. Then do the following:

Hit F5 to start debugging
Once the application has started, you should see two ports appear. Change the visibility of the second port (8443) to “public” via the right-click menu, then copy the address via the right-click menu. This will put an address like https://cuddly-capybara-q74jxp677rj24q7j-8443.app.github.dev/ into your clipboard, but with different random names and numbers.

Use that URL with the additional path /webhook/inspect as URL when configuring your webhook, e.g. when a work item is updated.

Once you update a work item, you should see that request come in to your Codespace and it should log the payload to the debug console.

The details: The code for inspecting

The first step in analyzing what happens when a webhook is executed is to examine the payload. The official documentation mentioned above suggests using a public service like Pastebin, but you may not be able to or want to use it. Instead, you can run the aforementioned small application to inspect the payload. The code is trivial:

1
2
3
4
5
6
7
8
9
var app = WebApplication.CreateBuilder().Build();

app.MapPost("/webhook/inspect", async (HttpRequest request) =>
{
    using var reader = new StreamReader(request.Body);
    var body = await reader.ReadToEndAsync();
    Console.WriteLine($"Received webhook data:\n{body}");
    return Results.Ok();
});

That’s it. It simply creates an endpoint that responds to a POST request and writes the content to the console. It’s nothing fancy, but it helps to understand what is shared to a webhook. Using our previous example, if this is called after a work item is updated, the abbreviated log would show something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
{
    "subscriptionId": "2f0622c6-fe08-4357-a9e2-788668ad37a7",
    "notificationId": 1,
    "id": "29e26c11-013d-451c-868c-f897a6400d2a",
    "eventType": "workitem.updated",
    "publisherId": "tfs",
    "message": {
        "text": "User Story #1 (Test with changed title) updated by Tobias Fenster\r\n(https://dev.azure.com/repro-publishing-issue/web/wi.aspx?pcguid=557fa63a-3fb6-460a-866b-cfcae8fedd66&id=1)",
        "html": "<a href=\"https://dev.azure.com/repro-publishing-issue/web/wi.aspx?pcguid=557fa63a-3fb6-460a-866b-cfcae8fedd66&amp;id=1\">User Story #1</a> (Test with changed title) updated by Tobias Fenster",
        "markdown": "[User Story #1](https://dev.azure.com/repro-publishing-issue/web/wi.aspx?pcguid=557fa63a-3fb6-460a-866b-cfcae8fedd66&id=1) (Test with changed title) updated by Tobias Fenster"
    },
    "detailedMessage": {
        "text": "User Story #1 (Test with changed title) updated by Tobias Fenster\r\n(https://dev.azure.com/repro-publishing-issue/web/wi.aspx?pcguid=557fa63a-3fb6-460a-866b-cfcae8fedd66&id=1)\r\n\r\n- New Title: Test with changed title\r\n",
        "html": "<a href=\"https://dev.azure.com/repro-publishing-issue/web/wi.aspx?pcguid=557fa63a-3fb6-460a-866b-cfcae8fedd66&amp;id=1\">User Story #1</a> (Test with changed title) updated by Tobias Fenster<ul>\r\n<li>New Title: Test with changed title</li></ul>",
        "markdown": "[User Story #1](https://dev.azure.com/repro-publishing-issue/web/wi.aspx?pcguid=557fa63a-3fb6-460a-866b-cfcae8fedd66&id=1) (Test with changed title) updated by Tobias Fenster\r\n\r\n* New Title: Test with changed title\r\n"
    },
    "resource": {
        "id": 2,
        "workItemId": 1,
        "rev": 2,
        "revisedBy": {
            "id": "72449079-0c93-6274-848f-7e657a28086d",
            "name": "Tobias Fenster <tfenster@4psbau.de>",
            "displayName": "Tobias Fenster",
            "url": "https://spsprodneu1.vssps.visualstudio.com/A5fc63d1b-ae70-4eb7-8657-56f682d08852/_apis/Identities/72449079-0c93-6274-848f-7e657a28086d",
            "_links": {
                "avatar": {
                    "href": "https://dev.azure.com/repro-publishing-issue/_apis/GraphProfile/MemberAvatars/aad.NzI0NDkwNzktMGM5My03Mjc0LTg0OGYtN2U2NTdhMjgwODZk"
                }
            },
            "uniqueName": "tfenster@4psbau.de",
            "imageUrl": "https://dev.azure.com/repro-publishing-issue/_apis/GraphProfile/MemberAvatars/aad.NzI0NDkwNzktMGM5My03Mjc0LTg0OGYtN2U2NTdhMjgwODZk",
            "descriptor": "aad.NzI0NDkwNzktMGM5My03Mjc0LTg0OGYtN2U2NTdhMjgwODZk"
        },
        "revisedDate": "9999-01-01T00:00:00Z",
        "fields": {
            "System.Rev": {
                "oldValue": 1,
                "newValue": 2
            },
            "System.AuthorizedDate": {
                "oldValue": "2025-07-27T14:03:59.473Z",
                "newValue": "2025-07-27T14:04:08.447Z"
            },
            "System.RevisedDate": {
                "oldValue": "2025-07-27T14:04:08.447Z",
                "newValue": "9999-01-01T00:00:00Z"
            },
            "System.ChangedDate": {
                "oldValue": "2025-07-27T14:03:59.473Z",
                "newValue": "2025-07-27T14:04:08.447Z"
            },
            "System.Watermark": {
                "oldValue": 1,
                "newValue": 2
            },
            "System.Title": {
                "oldValue": "Test",
                "newValue": "Test with changed title"
            }
        },
        "_links": {
            "self": {
                "href": "https://dev.azure.com/repro-publishing-issue/07afe328-1c60-4995-8420-0a70c400fc5d/_apis/wit/workItems/1/updates/2"
            },
            "workItemUpdates": {
                "href": "https://dev.azure.com/repro-publishing-issue/07afe328-1c60-4995-8420-0a70c400fc5d/_apis/wit/workItems/1/updates"
            },
            "parent": {
                "href": "https://dev.azure.com/repro-publishing-issue/07afe328-1c60-4995-8420-0a70c400fc5d/_apis/wit/workItems/1"
            },
            "html": {
                "href": "https://dev.azure.com/repro-publishing-issue/web/wi.aspx?pcguid=557fa63a-3fb6-460a-866b-cfcae8fedd66&id=1"
            }
        },
        "url": "https://dev.azure.com/repro-publishing-issue/07afe328-1c60-4995-8420-0a70c400fc5d/_apis/wit/workItems/1/updates/2",
        "revision": {
            "id": 1,
            "rev": 2,
            "fields": {
                "System.AreaPath": "repro",
                "System.TeamProject": "repro",
                "System.IterationPath": "repro",
                "System.WorkItemType": "User Story",
                "System.State": "New",
                "System.Reason": "New",
                "System.CreatedDate": "2025-07-27T14:03:59.473Z",
                "System.CreatedBy": "Tobias Fenster <tfenster@4psbau.de>",
                "System.ChangedDate": "2025-07-27T14:04:08.447Z",
                "System.ChangedBy": "Tobias Fenster <tfenster@4psbau.de>",
                "System.CommentCount": 0,
                "System.Title": "Test with changed title",
                "System.BoardColumn": "New",
                "System.BoardColumnDone": false,
                "Microsoft.VSTS.Common.StateChangeDate": "2025-07-27T14:03:59.473Z",
                "Microsoft.VSTS.Common.Priority": 2,
                "Microsoft.VSTS.Common.ValueArea": "Business",
                "WEF_3139F49F3CA94BDB98022BF3C1AF3733_Kanban.Column": "New",
                "WEF_3139F49F3CA94BDB98022BF3C1AF3733_Kanban.Column.Done": false
            },
            "multilineFieldsFormat": {},
            "_links": {
                "self": {
                    "href": "https://dev.azure.com/repro-publishing-issue/07afe328-1c60-4995-8420-0a70c400fc5d/_apis/wit/workItems/1/revisions/2"
                },
                "workItemRevisions": {
                    "href": "https://dev.azure.com/repro-publishing-issue/07afe328-1c60-4995-8420-0a70c400fc5d/_apis/wit/workItems/1/revisions"
                },
                "parent": {
                    "href": "https://dev.azure.com/repro-publishing-issue/07afe328-1c60-4995-8420-0a70c400fc5d/_apis/wit/workItems/1"
                }
            },
            "url": "https://dev.azure.com/repro-publishing-issue/07afe328-1c60-4995-8420-0a70c400fc5d/_apis/wit/workItems/1/revisions/2"
        }
    },
    "resourceVersion": "1.0",
    "resourceContainers": {
        "collection": {
            "id": "557fa63a-3fb6-460a-866b-cfcae8fedd66",
            "baseUrl": "https://dev.azure.com/repro-publishing-issue/"
        },
        "account": {
            "id": "5fc63d1b-ae70-4eb7-8657-56f682d08852",
            "baseUrl": "https://dev.azure.com/repro-publishing-issue/"
        },
        "project": {
            "id": "07afe328-1c60-4995-8420-0a70c400fc5d",
            "baseUrl": "https://dev.azure.com/repro-publishing-issue/"
        }
    },
    "createdDate": "2025-07-27T14:04:14.970126Z"
}

A few things to highlight:

The eventType in line 5 shows which type of event triggered the webhook call, in this case workitem.updated. You can find a full list across “Build and release”, “Pipeline”, “Code”, Service connection”, “Work item” and “Advanced security” events in the documentation
The revisedBy gives you a way to figure out who made the change (from line 21 onwards)
In my scenario, I am interested in fields that changed, which is why I would look at the fields from line 36 onwards, where I can see the old and new values for every changed field.

This will, of course, be different depending on the event you listen to, but that is why having a handy “inspector” is so useful. You just run the service, configure the webhook, and examine the payload to understand what’s coming in and how to make the most of it.

The details: The code for reacting to a work item update

Once you have a clear idea of what you want, it makes sense to create a model class so that you can interact with the payload in a strongly typed way. The easiest way nowadays, in my opionion, is to just ask GitHub Copilot to create it based on the payload you get through an inspection as explained above. I won’t bore you with the details as it is just a C# representation of the JSON payload above, but you can find it here if you want to take a look.

Handling the content would look something like this if you wanted to e.g. react to a changed title:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
app.MapPost("webhook/updatedWorkItem", async (HttpRequest request) =>
{
    Console.WriteLine("Updated work item received");
    using var reader = new StreamReader(request.Body);
    var body = await reader.ReadToEndAsync();
    var payload = JsonSerializer.Deserialize<AzDevOpsWebhookPayload>(body, new JsonSerializerOptions()
    {
        PropertyNameCaseInsensitive = true
    });
    if (payload == null)
    {
        return Results.BadRequest("Invalid payload");
    }

    if (payload.EventType == "workitem.updated" && payload.Resource?.Fields != null)
    {
        if (payload.Resource.Fields.TryGetValue("System.Title", out var titleField))
        {
            Console.WriteLine($"Work item updated: {payload.Resource.WorkItemId}, Old Title: {titleField.OldValue} --> New Title: {titleField.NewValue}");
        }
    }
    return Results.Ok();
});

Again, this reacts to a POST request (line 1), gets the body (lines 4 and 5) and deserializes it into the payload object (lines 6-9). Now we can work with the content:

Line 15 checks if the EventType is the right one and whether we have any changed fields
Line 17 tries to read the System.Title field
And then in line 19 we handle it, in this case only showing the old and new title in the logs.

With that code and the GitHub Codespace in place, you can play around, see what Azure DevOps is sending and react to it as your requirements demand.

The details: Deploying it to production, e.g. an AKS cluster

But how can we run this in production? Nowadays, chances are that you have a Kubernetes cluster somewhere in your reach. Deploying it there is quite easy and in my case with an Azure Kubernetes Service (AKS) cluster, it would look like this although it probably would work the same somewhere else:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
kind: Namespace
apiVersion: v1
metadata:
  name: azdevops-webhook-explorer-linux
  labels:
    name: azdevops-webhook-explorer-linux
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: azdevops-webhook-explorer
  namespace: azdevops-webhook-explorer-linux
spec:
  replicas: 1
  selector:
    matchLabels:
      app: azdevops-webhook-explorer
  template:
    metadata:
      labels:
        app: azdevops-webhook-explorer
    spec:
      containers:
        - name: azdevops-webhook-explorer
          image: tobiasfenster/azdevops-webhook-explorer:latest
          imagePullPolicy: Always
          resources:
            limits:
              memory: "1024Mi"
              cpu: "1000m"
      nodeSelector:
        kubernetes.io/os: linux
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: azdevops-webhook-explorer-ingress
  namespace: azdevops-webhook-explorer-linux
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /webhook/$1

spec:
  ingressClassName: nginx
  rules:
    - host: fps-alpaca.westeurope.cloudapp.azure.com
      http:
        paths:
          - backend:
              service:
                name: azdevops-webhook-explorer
                port:
                  number: 8080
            path: /azdevops-webhook-explorer/(.*)
            pathType: Prefix
  tls:
    - hosts:
        - fps-alpaca.westeurope.cloudapp.azure.com
---
apiVersion: v1
kind: Service
metadata:
  name: azdevops-webhook-explorer
  namespace: azdevops-webhook-explorer-linux
spec:
  selector:
    app: azdevops-webhook-explorer
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

First we have the Namespace, kind of the “folder” to keep the kubernetes resources together, shown in lines 1-6. The we have the Deployment, which among others defines the container image to use (line 25), the number of instances or replicas (line 14) and the maximum reosurces to use (lines 27-30). Then we have the Ingress which is a way to receive traffic from the outside by defining a host (lines 45 and 57, the latter one for the TLS setup) and path on which to react (line 53). Because of the annotation in line 40, the incoming request to /azdevops-webhook-explorer/ is redirected to /webhook/ and the rest of the incoming path is appended through the (.*) in the path and the $1 in the annotation. You can also see in lines 49-52 that the request is sent to a service called azdevops-webhook-explorer on port 8080. Whic brings us to the last part, the Service which is connected to the Deployment via the selector in lines 65 and 66 where you can see the corresponding matchLabels in line 16 and 17.

The Dockerfile to create an image out of the code explained above is very standard, but if you want to take a look, you can find it here.

This should hopefully give you an idea how you can easily develop such a webhook using a GitHub Codespace and how you can as easily deploy it to a Kubernetes environment like the Azure Kubernetes Service.

The underlying requirement is that we have two boards and work items are assigned to each of them based on the area path. Once a work item has reached a certain column in the first board, it should automatically be moved to the second one, which means the area path has to change. ↩

Docker Offload for blazing fast AI development

Thu, 10 Jul 2025 08:00:24 +0000

Maybe you have seen the announcement, Docker just introduced Docker Offload, an offering to run especially (but not only) GPU-reliant AI workloads in a secure, managed cloud environment. It works seamlessly through the familiar UI of Docker Desktop or the Docker CLI. Docker has shared a very good introduction and samples, but I want to show you how my little sample application I introduced in a previous blog post can be used in Offload as well.

The TL;DR

If you want to follow along quickly, here are the steps

Clone my repository at http://github.com/tfenster/dotnet-model-runner/
Go to the .devcontainer folder and run docker compose up -d
Once it has pulled and started everything (should be blazing fast thanks to the Offloud cloud inftrastructure), use the Dev Containers: Attach to Running Container... action of the VS Code “Dev Containers” extension. To learn more why the usual Dev Containers: Clone Repository in Container Volume action doesn’t work (yet), check the details below
Wait until everything has loaded (“installing server”, “starting server”)
Do two steps which typically would happen automatically through the devcontainer setup, but for now doesn’t for the reasons explained below:
- Install the “C# Dev Kit” extension in the devcontainer
- Run dotnet dev-certs https in the devcontainer
Start debugging by hitting F5

With that you should get my little chat application to talk to the LLMs via the Docker Model Runner (DMR), but this time not locally, but in the Offload cloud infrastructure. The whole interaction, debugging, executing etc works exactly the same, but (at least for me) at lot more performant than locally. A really smooth experience.

The details: Models in docker compose and how to reference them

I want to point out one feature in docker compose that is relatively new and lets you define DMR AI models as components of your application like other (containerized) parts. It looks like this in my example application:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
services:
  dev:
    image: "mcr.microsoft.com/devcontainers/dotnet:1-9.0-bookworm"
    models:
      - smollm2
      - phi4
    volumes:
      - ..:/workspace:cached
    command: /bin/sh -c "while sleep 1000; do :; done"

models:
  smollm2:
    model: ai/smollm2
  phi4:
    model: ai/phi4

In lines 11-15 you see how DMR models can be easily defined and lines 5 and 6 show how they are referenced by applications. This leads to among others an environment variable SMOLLM2_URL being populated with the URL where the models are available from within the container. If you want to learn more about how to configure this and other elements, check the [official docs][docs]. Now in my client application, I need a reference to the models. As I am using the OpenAI .NET API library, that is just a small piece of code:

1
2
3
4
5
6
7
8
var baseUrl = Environment.GetEnvironmentVariable("SMOLLM2_URL")
    ?? throw new InvalidOperationException("SMOLLM2_URL environment variable is not set.");
var endpoint = baseUrl.Replace("/v1", "/llama.cpp/v1");
var options = new OpenAIClientOptions
{
    Endpoint = new Uri(endpoint),
};
return new OpenAIClient(new ApiKeyCredential("unused"), options);

You can see first the retrieval of the environment variable, then I small change to the URL to get the expected endpoint and then the configuration of the OpenAIClient.

The details: Why so complicated (when it comes to the devcontainer)

Now, why can’t we just clone the repo into a volume and run the container on that? The model configuration as explained above is only supported with docker compose v2.28 and newer. However, VS Code starts what they call a “bootstrap container” to set up the actual devcontainer and in this case also running the docker compose commands and unfortunately doesn’t provide a way to override or configure that. Because that container has docker compose v2.17, the model element doesn’t work and startup fails.

I have added a comment that a configuration option for this bootstrap container would be very useful, but until that happens, a workaround like explained above is needed.

What are FoDs or how to run IIS in a Windows Server 2025 Nano Server container

Sun, 22 Jun 2025 17:43:40 +0000

Windows containers are often considered too heavy and resource-hungry for the highly dynamic worlds of microservices and edge computing. And, to be honest, there is some truth to that. However, Microsoft has made significant progress with Nano Server container images and has reduced the size and footprint of these images through multiple iterations. We have seen extracted images of less than 300 MB, and while this figure has increased again with Windows Server 2025, the images are still less than 500 MB. This is not tiny by any means in the Linux world, but compared to a Server Core container image, which was ~5 GB extracted and is now ~7.5 GB with the latest updates, it is quite small. The main disadvantage of Nano Server was that it often didn’t provide exactly what you needed, so in some worst-case scenarios, a simple service or feature could force you to switch to Server Core. Fortunately, this has changed with FoDs (‘Features on Demand’), which allow you to install specific features on demand - as the name suggests. Microsoft’s announcement provides some details and an example, but I would like to give you another one below.

The TL;DR

To see how it works locally, do the following:

Clone my sample repo at https://github.com/tfenster/FoDs
Make sure you have Docker Desktop installed and configured for Windows containers
Execute the following command in the directory where you cloned the repo: docker build -t nanoserver-iis .. This will build an image that uses the IIS FoD to enable IIS in Nano Server.
Run a container with that image and expose the default IIS port 80 on port 8080 of your localhost by using the following command: docker run -p 8080:80 -ti nanoserver-iis.
You can then access your IIS running in a Nano Server container at http://localhost:8080

While this runs only locally, it is a crucial first step to bringing a better overall experience to e.g. Azure Kubernetes Service when running Windows workloads. Once Windows Server 2025 is supported there, I’ll follow up again.

The details: The Dockerfile

The Dockerfile to make this happen looks like this

1
2
3
4
5
6
7
FROM mcr.microsoft.com/windows/nanoserver:ltsc2025
WORKDIR /install
COPY install_iis.cmd .
USER ContainerAdministrator
RUN install_iis.cmd
USER ContainerUser
WORKDIR /inetpub

We start from Windows Server 2025 Nano Server (line 1) which is the first to support FoDs. We then copy the install script in and run it (lines 2-5). Note that we run it as ContainerAdministrator (line 4) because elevated permissions are required. Then, we switch back to the non-privileged ContainerUser and go to the inetpub folder (lines 6 and 7), which contains the IIs files.

The details: The install script

The install script is even shorter and looks like this

1
2
3
4
5
DISM /Online /Add-Capability /CapabilityName:Microsoft.NanoServer.IIS /NoRestart
if errorlevel 3010 (
    echo The specified optional feature requested a reboot which was suppressed.
    exit /b 0
)

The first line installs the FoD and uses the /NoRestart parameter to ensure that no restart occurs. As this would cause an error, lines 2–5 are needed to handle it properly.

There are no additional details, but the above-mentioned announcement provides some more background information. This may not seem like a big deal, but once Microsoft adds more FoDs for us to choose from, it has the potential to unlock many scenarios in Nano Server and lead to much smaller container images.

Docker AI models are available on Windows laptops now

Wed, 28 May 2025 17:45:33 +0000

Docker recently revealed a new feature of Docker Desktop: The Docker Model Runner. Initially it was only available on Mac and Windows with Nvidia GPUs, but with the upcoming release 4.42 of Docker Desktop, also Windows laptops with Qualcomm / ARM GPUs are supported. As I have such a laptop and got a chance to test a pre-release version of 4.42, I could finally join in on the fun.

The TL;DR

If you have a compatible laptop and Docker Desktop 4.42, here’s an example of what you can do:

Note that you have to enable the Model Runner either through Settings -> Beta Features where you also have to enable the GPU-backed inference. Enabling the Model Runner itself is also possible from the CLI with docker desktop enable model-runner, but enabling the GPU support only works from the UI for now. Also note that only models with a tag *-Q4_0 make use of it, something I learned only afterwards :)

The details: interacting with a model

As you can see, I used a SmolLM2 model with 135 million parameters, so it’s a very lightweight model. This is visible through the docker command: I executed docker model run ai/smollm2:135M-Q2_K, so the smollm2 indicates the model’s name and the 135M shows that it is the 135M parameter variation. With a simple docker model run, you can get an interactive chat session with the model. If you want to learn more about that, I recommend the official Docker docs.

To see which models are available, you can go to the AI section of the Docker Hub. There, you will find an ever-expanding list of models, each with a description, the different variations and some performance benchmarks to help you decide which model to use.

The details: using it in a .NET Blazor application

While that is a nice demo case, we probably want to use it for application development. Therefore, I have also created a small example of how it works from a .NET application using the OpenAI package. The most relevant part is probably the endpoint where the model runner is reachable or at least it took me the longest to figure that out and put it correctly into my code. It is available from within a container at http://model-runner.docker.internal or from the host at http://localhost:11434. If you want a different port, you can also change that using docker desktop enable model-runner --tcp <port> or through the settings UI. For example to query for the available models, you could make an HTTP call like this

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
GET http://localhost:11434/engines/llama.cpp/v1/models

{
  "object": "list",
  "data": [
    {
      "id": "ai/smollm2:135M-Q2_K",
      "object": "model",
      "created": 1745936469,
      "owned_by": "docker"
    },
    {
      "id": "ai/deepseek-r1-distill-llama",
      "object": "model",
      "created": 1742905580,
      "owned_by": "docker"
    }
  ]
}

To integrate this into a simple Blazor application that allows us to chat with the models, we can configure the OpenAI client to use that endpoint. I am (of course ;)) developing this in a devcontainer, so I can use the internal URL e.g. like this

1
2
3
4
5
6
7
8
builder.Services.AddSingleton(sp =>
{
    var options = new OpenAIClientOptions
    {
        Endpoint = new Uri("http://model-runner.docker.internal/engines/llama.cpp/v1"),
    };
    return new OpenAIClient(new ApiKeyCredential("unused"), options);
});

When we want to have a chat interaction with the model, we use something like this:

1
2
3
4
5
6
7
8
9
10
11
12
var chatClient = OpenAIClient.GetChatClient(selectedModel);          
cancellationTokenSource = new CancellationTokenSource();

var messages = new[] { ChatMessage.CreateUserMessage(userInput) };
status = "Generating response...";	
await foreach (var message in chatClient.CompleteChatStreamingAsync(messages, null, cancellationTokenSource.Token))
{
    foreach (var update in message.ContentUpdate) {
        response += update.Text;
    }
    StateHasChanged();
}

You can check the full result on this Github repo and run it in a devcontainer. It should look like this, depending on which models you have pulled:

I hope this gives you an idea of why I’m so excited about this new Docker Desktop capability! Give it a try once it is officially released and let me know how you like it.