Issue Description

Repro step:

1. Boot system without EBOX connected normally
2. connect EBOX with RTX3060
3. Wait 5s
4. System BSOD

Issue CND if EBOX is connected when system boot.

Once EBOX is connected from boot, further hotplug will CND.

Analysis

4: kd> k
 # Child-SP          RetAddr               Call Site
00 ffff998f`fe34eb38 fffff805`920d6ce5     nt!KeBugCheckEx [minkernel\ntos\ke\amd64\procstat.asm @ 140]
01 ffff998f`fe34eb40 fffff805`9203425f     nt!MiSystemFault+0x735 [minkernel\ntos\mm\mmfault.c @ 4893]
02 ffff998f`fe34ec30 fffff805`92480ccb     nt!MmAccessFault+0x2ff [minkernel\ntos\mm\mmfault.c @ 8472]
03 ffff998f`fe34eda0 fffff805`2a550336     nt!KiPageFault+0x38b [minkernel\ntos\ke\amd64\trap.asm @ 1552]
04 ffff998f`fe34ef38 fffff805`2a5501ab     amdacpbtacx!CAfdBtI2SInterface::REG_READ+0x2a [c:\constructicon\builds\gfx\five\24.10\drivers\acp\acx\amdacpbtacx\src\acpbtacx_i2sinterface.cpp @ 447]
05 ffff998f`fe34ef40 fffff805`2a59a111     amdacpbtacx!CAfdBtI2SInterface::EnableI2SControllerInstance+0x217 [c:\constructicon\builds\gfx\five\24.10\drivers\acp\acx\amdacpbtacx\src\acpbtacx_i2sinterface.cpp @ 329]
06 ffff998f`fe34efa0 fffff805`23d1a7b7     amdacpbtacx!AfdBt_EvtDevicePrepareHardware+0x199 [c:\constructicon\builds\gfx\five\24.10\drivers\acp\acx\amdacpbtacx\src\acpbtacx_device.cpp @ 475]
07 ffff998f`fe34f050 fffff805`23ccb191     Wdf01000!FxPnpDevicePrepareHardware::InvokeClient+0x27 [minkernel\wdf\framework\shared\irphandlers\pnp\pnpcallbacks.cpp @ 447]
08 ffff998f`fe34f0a0 fffff805`23cf44a4     Wdf01000!FxPrePostCallback::InvokeStateful+0xa5 [minkernel\wdf\framework\shared\irphandlers\pnp\cxpnppowercallbacks.cpp @ 291]
09 (Inline Function) --------`--------     Wdf01000!FxPnpDevicePrepareHardware::Invoke+0x51 [minkernel\wdf\framework\shared\irphandlers\pnp\pnpcallbacks.cpp @ 421]
0a ffff998f`fe34f0e0 fffff805`23d41b2c     Wdf01000!FxPkgPnp::PnpPrepareHardware+0xf8 [minkernel\wdf\framework\shared\irphandlers\pnp\pnpstatemachine.cpp @ 3596]
0b ffff998f`fe34f130 fffff805`23d08bc1     Wdf01000!FxPkgPnp::PnpEventStartingFromStopped+0x1c [minkernel\wdf\framework\shared\irphandlers\pnp\pnpstatemachine.cpp @ 2491]
0c ffff998f`fe34f160 fffff805`23d088d1     Wdf01000!FxPkgPnp::PnpEnterNewState+0x125 [minkernel\wdf\framework\shared\irphandlers\pnp\pnpstatemachine.cpp @ 1240]
0d ffff998f`fe34f200 fffff805`23d085a2     Wdf01000!FxPkgPnp::PnpProcessEventInner+0xd1 [minkernel\wdf\framework\shared\irphandlers\pnp\pnpstatemachine.cpp @ 1154]
0e ffff998f`fe34f280 fffff805`23cfca02     Wdf01000!FxPkgPnp::_PnpProcessEventInner+0x32 [minkernel\wdf\framework\shared\irphandlers\pnp\pnpstatemachine.cpp @ 983]
0f ffff998f`fe34f2b0 fffff805`23cfc8e5     Wdf01000!FxEventQueue::EventQueueWorker+0x9a [minkernel\wdf\framework\shared\irphandlers\pnp\eventqueue.cpp @ 279]
10 ffff998f`fe34f300 fffff805`9203b56a     Wdf01000!FxWorkItemEventQueue::_WorkItemCallback+0x25 [minkernel\wdf\framework\shared\irphandlers\pnp\km\eventqueuekm.cpp @ 114]
11 ffff998f`fe34f330 fffff805`92143370     nt!IopProcessWorkItem+0x45a [minkernel\ntos\io\iomgr\misc.c @ 1616]
12 ffff998f`fe34f3c0 fffff805`9225a5fa     nt!ExpWorkerThread+0x2a0 [minkernel\ntos\ex\worker.c @ 4303]
13 ffff998f`fe34f570 fffff805`92472c94     nt!PspSystemThreadStartup+0x5a [minkernel\ntos\ps\psexec.c @ 11878]
14 ffff998f`fe34f5c0 00000000`00000000     nt!KiStartSystemThread+0x34 [minkernel\ntos\ke\amd64\threadbg.asm @ 87]

The REG_READ is simple, it uses a private field m_pRegAddrStart , plus an offset from parameter _Offset . This is the address we fault on.

Looking into it you see the m_pRegAddrStart itself is invalid page.

4: kd> dx -id 0,0,ffffdf0e70548040 -r1 ((amdacpbtacx!CAfdBtI2SInterface *)0xffffdf0e7a17d440)
((amdacpbtacx!CAfdBtI2SInterface *)0xffffdf0e7a17d440)                 : 0xffffdf0e7a17d440 [Type: CAfdBtI2SInterface *]
    [=0xfffff8052a568b18] m_Instance       : 0xffffdf0e7a17d440 [Type: CAfdBtI2SInterface *]
    [+0x000] m_pRegAddrStart  : 0xffff958f6836f000 : Unable to read memory at Address 0xffff958f6836f000 [Type: unsigned char *]
    [+0x008] m_ControllerInstance : 0x3 [Type: unsigned long]
    [+0x00c] m_ClockMode      : 0x2 [Type: unsigned long]
    [+0x010] m_AcpRevisionID  : 0x70 [Type: unsigned char]
4: kd> !pte 0xffff958f6836f000
                                           VA ffff958f6836f000
PXE at FFFFF97CBE5F2958    PPE at FFFFF97CBE52B1E8    PDE at FFFFF97CA563DA08    PTE at FFFFF94AC7B41B78
contains 0A00000647026863  contains 0A00000647027863  contains 0A0000012FB7C863  contains 0000000000000000
pfn 647026    ---DA--KWEV  pfn 647027    ---DA--KWEV  pfn 12fb7c    ---DA--KWEV  not valid

If we trace back in code you see this field is defined and assigned by:

So it is assigned to the constructor parameter. Dive further back the constructor is called only in AfdBt_EvtBusDeviceAdd .

PAGED_CODE_SEG
NTSTATUS
AfdBt_EvtBusDeviceAdd(
    _In_    WDFDRIVER _Driver,
    _Inout_ PWDFDEVICE_INIT _DeviceInit
)
{
    PAGED_CODE();

    NTSTATUS Status = STATUS_SUCCESS;

    DBG_TRACE_STATUS();

    /* Initialize the pnpPowerCallbacks structure. */
    WDF_PNPPOWER_EVENT_CALLBACKS pnpPowerCallbacks;
    WDF_PNPPOWER_EVENT_CALLBACKS_INIT(&pnpPowerCallbacks);
    pnpPowerCallbacks.EvtDevicePrepareHardware = AfdBt_EvtDevicePrepareHardware;
    pnpPowerCallbacks.EvtDeviceReleaseHardware = AfdBt_EvtDeviceReleaseHardware;
    pnpPowerCallbacks.EvtDeviceD0Entry = AfdBt_EvtDeviceD0Entry;
    pnpPowerCallbacks.EvtDeviceD0EntryPostInterruptsEnabled = AfdBt_EvtDeviceD0EntryPostInterruptsEnabled;
    pnpPowerCallbacks.EvtDeviceD0ExitPreInterruptsDisabled = AfdBt_EvtDeviceD0ExitPreInterruptsDisabled;
    pnpPowerCallbacks.EvtDeviceD0Exit = AfdBt_EvtDeviceD0Exit;
    WdfDeviceInitSetPnpPowerEventCallbacks(_DeviceInit, &pnpPowerCallbacks);

...

    /* Tell the framework to set the SurpriseRemovalOK in the DeviceCaps so
    * that you don't get the popup in usermode (on Win2K) when you surprise
    * remove the device.
    */
    WDF_DEVICE_PNP_CAPABILITIES pnpCaps;
    WDF_DEVICE_PNP_CAPABILITIES_INIT(&pnpCaps);
    pnpCaps.SurpriseRemovalOK = WdfTrue;
    WdfDeviceSetPnpCapabilities(AfdBtDevice, &pnpCaps);

    /* Get the AcpBus Interface */
    CAfdBtAcpBusPciInterface::setInstance(new CAfdBtAcpBusPciInterface(pDevCtx, AcpClientTypeBt)); /*Child PDO is BT*/
    Status = CAfdBtAcpBusPciInterface::getInstance()->Initialize();
    ACP_STATUS_CHECK(Status, "CAfdBtAcpBusPciInterface::Initialize()", Exit);

    /* Initialize I2S class interface*/
    CAfdBtI2SInterface::setInstance(new CAfdBtI2SInterface(CAfdBtAcpBusPciInterface::getInstance()->m_pRegMap));

This is in EvtBusDeviceAdd , but when eBox hot plug, OS called only EvtDevicePrepareHardware .

Sequence of events

When 1st boot without eBox. OS calls the sequence:

1. DriverEntry
   1. Calls WDF_DRIVER_CONFIG_INIT(&wdfCfg, AfdBt_EvtBusDeviceAdd) , provide EvtBusDeviceAdd callback to OS.
2. WDF calls EvtBusDeviceAdd

   1. Creates PNPPOWERCALLBACK , provide WDF with a series of callbacks (Key point is EvtDevicePrepareHardware),and calls WdfDeviceInitSetPnpPowerEventCallbacks .

          WDF_PNPPOWER_EVENT_CALLBACKS pnpPowerCallbacks;
          WDF_PNPPOWER_EVENT_CALLBACKS_INIT(&pnpPowerCallbacks);
          pnpPowerCallbacks.EvtDevicePrepareHardware = AfdBt_EvtDevicePrepareHardware;
          pnpPowerCallbacks.EvtDeviceReleaseHardware = AfdBt_EvtDeviceReleaseHardware;
          pnpPowerCallbacks.EvtDeviceD0Entry = AfdBt_EvtDeviceD0Entry;
          pnpPowerCallbacks.EvtDeviceD0EntryPostInterruptsEnabled = AfdBt_EvtDeviceD0EntryPostInterruptsEnabled;
          pnpPowerCallbacks.EvtDeviceD0ExitPreInterruptsDisabled = AfdBt_EvtDeviceD0ExitPreInterruptsDisabled;
          pnpPowerCallbacks.EvtDeviceD0Exit = AfdBt_EvtDeviceD0Exit;
          WdfDeviceInitSetPnpPowerEventCallbacks(_DeviceInit, &pnpPowerCallbacks);

   2. Create a new CAfdBtI2SInterface , feed it with the register map address, and set it as static instance.

3. WDF calls EvtPrepareHardware
   1. We write some register based on the address in CAfdBtI2SInterface.m_pRegAddrStart .

So far, so good. Problems start occuring when we plug in eBox.

Since eBox is a PCIE hardware device, I’m guessing PnP has some rebalance to do. In fact, if we look at the device tree in dump, you see ACP has just went through this process.

DevNode 0xffffdf0e705a7b30 for PDO 0xffffdf0e743ed060
  InstancePath is "PCI\VEN_1022&DEV_15E2&SUBSYS_15E21022&REV_70\4&35fe04f8&0&0541"
  ServiceName is "amdacpbus"
  State = DeviceNodeStarted (0x30a)
  Previous State = DeviceNodeRestartCompletion (0x30d)
  DevNode 0xffffdf0e844609a0 for PDO 0xffffdf0e820a5410
    InstancePath is "ACP\DEVTYPE_0001&DEVREV_0000&VEN_1022&DEV_15E2&SUBSYS_102215E2&REV_70\5&105e5341&1&01"
    State = DeviceNodeRestartCompletion (0x30d)
    Previous State = DeviceNodeStartPending (0x307)
  DevNode 0xffffdf0e8445b9a0 for PDO 0xffffdf0e820bb8f0
    InstancePath is "ACP\DEVTYPE_0004&DEVREV_0001&VEN_1022&DEV_15E2&SUBSYS_102215E2&REV_70\5&105e5341&1&05"
    ServiceName is "AMDAcpBtAudioService"
    TargetDeviceNotify List - f 0xffff9d0a2f6eb190  b 0xffff9d0a2f6eb190
    State = DeviceNodeStartPending (0x307)
    Previous State = DeviceNodeStopped (0x30c)

When we plug in the eBox, WDF only calls EvtPrepareHardware . Where it expects us to reinitialize the device. But we previously stored CAfdBtI2SInterface.m_pRegAddrStart as a static instance. We are never released or unloaded, so we used a wrong value and bugchecked.

According to MSFT, we should “Map physical memory addresses to VA” in EvtPrepareHardware instead of in EvtBusDeviceAdd .

See also:

https://learn.microsoft.com/en-us/windows-hardware/drivers/wdf/a-user-plugs-in-a-device

https://learn.microsoft.com/en-us/windows-hardware/drivers/wdf/the-pnp-manager-redistributes-system-resources