Ticket #32404

VM1にCPUを1個しか割り当てずにVM1でFreeBSDを起動しようとすると、NICがリンクアップしない

Open Date: 2013-11-05 00:10 Last Update: 2016-07-24 21:52

Reporter:
Owner:
Type:
Status:
Closed
Component:
(None)
MileStone:
(None)
Priority:
5 - Medium
Severity:
5 - Medium
Resolution:
Works For Me
File:
2

Details

re0: link state changed to DOWN

障害発生リビジョン: r118

Ticket History (3/26 Histories)

2013-11-05 00:10 Updated by: yuichi_xy
  • New Ticket "VM1にCPUを1個しか割り当てずにVM1でFreeBSDを起動しようとすると、NICがリンクアップしない" created
2013-11-05 22:03 Updated by: yuichi_xy
Comment

Linuxの場合は、CPUが1個のみでもNICがリンクアップする。

[    6.406644] r8169 0000:05:00.0: firmware: agent loaded rtl_nic/rtl8168e-3.fw into memory
[    6.474149] r8169 0000:05:00.0: eth0: link down
[    6.476086] ADDRCONF(NETDEV_UP): eth0: link is not ready
[    7.028587] r8169 0000:05:00.0: eth0: link up
2013-11-06 22:50 Updated by: yuichi_xy
Comment

FreeBSDは物理ディスティネーションモード固定であるようだ。

usr/src/sys/x86/x86/msi.c

#define INTEL_ADDR(msi)                                                 \
        (MSI_INTEL_ADDR_BASE | (msi)->msi_cpu << 12 |                   \
            MSI_INTEL_ADDR_RH_OFF | MSI_INTEL_ADDR_DM_PHYSICAL)
#define INTEL_DATA(msi)                                                 \
        (MSI_INTEL_DATA_TRGREDG | MSI_INTEL_DATA_DELFIXED | (msi)->msi_vector)
2013-11-07 23:25 Updated by: yuichi_xy
Comment

VM1にCPUを1個のみ割り当てた場合は、リンクアップしました。

Trying to mount root from ufs:/dev/ada0p2 [rw]...
re0: link state changed to DOWN
re0: link state changed to UP
2013-11-07 23:36 Updated by: yuichi_xy
Comment

yuichi_xy への返信

VM1にCPUを1個のみ割り当てた場合は、リンクアップしました。

VM0の間違いです。

VM0にCPUを1個のみ割り当てた場合は、リンクアップしました。

{{{ Trying to mount root from ufs:/dev/ada0p2 rw... re0: link state changed to DOWN re0: link state changed to UP }}}

2013-11-07 23:39 Updated by: yuichi_xy
Comment

X Window(gdm)を起動しないようにしても、 VM1にCPUを1個のみ割り当てた場合は、NICがリンクアップしませんでした。

Trying to mount root from ufs:/dev/ada0p2 [rw]...
atrtc0: WARNING: Battery failure indication
Invalid time in real time clock.
Check and reset the date immediately!
re0: link state changed to DOWN
2013-11-09 21:31 Updated by: yuichi_xy
Comment

VM1にCPUを2個割り当てて、FreeBSDを起動してみた。 NICはMSI-Xが有効になっていた。

05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 P
CI Express Gigabit Ethernet Controller (rev 06)
        Subsystem: ASRock Incorporation Motherboard (one of many)
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Step
ping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- 
<MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 255
        Region 0: I/O ports at d000
        Region 2: Memory at f0004000 (64-bit, prefetchable)
        Region 4: Memory at f0000000 (64-bit, prefetchable)
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3h
ot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [70] Express (v2) Endpoint, MSI 01
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L
1 <64us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupporte
d-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 4096 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPe
nd-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L
0 unlimited, L1 <64us
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive
- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF
 Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OB
FF Disabled
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedC
ompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-,
 EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizat
ionRequest-
        Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
                Vector table: BAR=4 offset=00000000
                PBA: BAR=4 offset=00000800
        Capabilities: [d0] Vital Product Data
                Not readable

SATAはMSI

06:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) (prog-if 01 [AHCI 1.0])
        Subsystem: ASRock Incorporation Motherboard
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 255
        Region 0: I/O ports at c050
        Region 1: I/O ports at c040
        Region 2: I/O ports at c030
        Region 3: I/O ports at c020
        Region 4: I/O ports at c000
        Region 5: Memory at f7800000 (32-bit, non-prefetchable)
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee06000  Data: 0032
        Capabilities: [78] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [80] Express (v2) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s, Latency L0 <512ns, L1 <2us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABC, TimeoutDis+, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
2013-11-09 21:42 Updated by: yuichi_xy
Comment

yuichi_xy への返信

VM1にCPUを2個割り当てて、FreeBSDを起動してみた。 NICはMSI-Xが有効になっていた。

Capabilities: b0 MSI-X: Enable+ Count=4 Masked- Vector table: BAR=4 offset=00000000 PBA: BAR=4 offset=00000800 Capabilities: d0 Vital Product Data Not readable }}}

メモリ空間のレジスタにも、それっぽい値が設定されていた。

# dd if=/dev/mem bs=1 count=32 iseek=0xf0000000 | hexdump 32+0 records in 32+0 records out 32 bytes transferred in 0.000216 secs (148143 bytes/sec) 0000000 4000 fee0 0000 0000 003b 0000 0000 0000 0000010 e066 11ae d189 7d1a dca2 87ae 0001 0000 0000020

2013-11-17 21:10 Updated by: yuichi_xy
Comment

CPUを1個しか割り当てないVM1でLinuxを起動した場合は、NICも使え、xHCIも使える。

2013-11-17 21:50 Updated by: yuichi_xy
Comment

CPUを2個割り当てたVM1でFreeBSDを起動して、プロセスをツリー表示してみた。 どこかで、引っかかってしまっているような兆候はみられなかった。

# pstree 0 -w
-+= 00000 root [kernel]
 |-+= 00001 root /sbin/init --  
 | |--= 00127 root adjkerntz -i 
 | |--= 00483 root /sbin/devd   
 | |--= 00607 root /usr/sbin/syslogd -s
 | |-+= 00708 root /usr/sbin/sshd
 | | \-+= 00764 root sshd: yuichi [priv] (sshd)
 | |   \-+- 00767 yuichi sshd: yuichi@pts/0 (sshd)
 | |     \-+= 00768 yuichi -sh (sh)
 | |       \-+= 00846 root su   
 | |         \-+= 00847 root _su (csh)
 | |           \-+= 00867 root pstree 0 -w
 | |             \--- 00868 root ps -axwwo user,pid,ppid,pgid,command
 | |--= 00715 root sendmail: accepting connections (sendmail)
 | |--= 00718 smmsp sendmail: Queue runner@00:30:00 for /var/spool/clientmqueue
 | |--= 00722 root /usr/sbin/cron -s
 | |--- 00755 root /usr/libexec/getty Pc ttyv0
 | |--- 00756 root /usr/libexec/getty Pc ttyv1
 | |--- 00757 root /usr/libexec/getty Pc ttyv2
 | |--- 00758 root /usr/libexec/getty Pc ttyv3
 | |--- 00759 root /usr/libexec/getty Pc ttyv4
 | |--- 00760 root /usr/libexec/getty Pc ttyv5
 | |--- 00761 root /usr/libexec/getty Pc ttyv6
 | |--- 00762 root /usr/libexec/getty Pc ttyv7
 | \--= 00763 root /usr/libexec/getty std.115200 ttyu0
 |--- 00002 root [sctp_iterator]
 |--- 00003 root [xpt_thrd]
 |--- 00004 root [pagedaemon]
 |--- 00005 root [vmdaemon]
 |--- 00006 root [pagezero]
 |--- 00007 root [bufdaemon]
 |--- 00008 root [syncer]
 |--- 00009 root [vnlru]
 |--- 00010 root [audit]
 |--- 00011 root [idle]
 |--- 00012 root [intr]
 |--- 00013 root [geom]
 |--- 00014 root [usb]
 \--- 00015 root [softdepflush]
2013-11-17 22:53 Updated by: yuichi_xy
Comment

top -Pコマンドで各CPUの状態を見てみたが、どちらのCPUでもプロセスが実行されていた。

last pid:  1152;  load averages:  0.00,  0.00,  0.00    up 0+01:21:04  22:52:31
22 processes:  1 running, 21 sleeping
CPU 0:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 1:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
Mem: 14M Active, 18M Inact, 278M Wired, 263M Buf, 4932M Free
Swap: 4096M Total, 4096M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
  767 yuichi        1  20    0 72136K  5848K select  0   0:08  0.00% sshd
  847 root          1  20    0 17564K  3684K pause   0   0:05  0.00% csh
  715 root          1  20    0 20340K  4568K select  1   0:02  0.00% sendmail
 1152 root          1  20    0 16596K  2392K CPU0    0   0:01  0.00% top
  768 yuichi        1  23    0 14536K  2300K wait    0   0:01  0.00% sh
  764 root          1  20    0 72136K  5808K select  1   0:01  0.00% sshd
  846 yuichi        1  20    0 45324K  2344K wait    0   0:00  0.00% su
  607 root          1  20    0 12076K  1744K select  1   0:00  0.00% syslogd
  722 root          1  20    0 14176K  1852K nanslp  0   0:00  0.00% cron
  757 root          1  20    0 12084K  1716K nanslp  0   0:00  0.00% getty
  756 root          1  20    0 12084K  1716K nanslp  0   0:00  0.00% getty
  761 root          1  20    0 12084K  1716K nanslp  1   0:00  0.00% getty
  762 root          1  20    0 12084K  1716K nanslp  0   0:00  0.00% getty
  760 root          1  20    0 12084K  1716K nanslp  0   0:00  0.00% getty
  759 root          1  20    0 12084K  1716K nanslp  1   0:00  0.00% getty
  755 root          1  20    0 12084K  1716K nanslp  1   0:00  0.00% getty
  758 root          1  20    0 12084K  1716K nanslp  1   0:00  0.00% getty
  763 root          1  52    0 12084K  1676K ttyin   1   0:00  0.00% getty
  483 root          1  20    0 10376K  4432K select  1   0:00  0.00% devd
  718 smmsp         1  20    0 20340K  4636K pause   1   0:00  0.00% sendmail
  708 root          1  20    0 46876K  5332K select  0   0:00  0.00% sshd
  127 root          1  52    0  9952K  1480K pause   1   0:00  0.00% adjkerntz
2013-11-17 23:39 Updated by: yuichi_xy
Comment

以下のページを参考に、CPUごとの割り込み発生数が取得できないか試みてみたが、interrupt_sourcesやintrcntはCPUごとに値を管理しているわけではなさそう。

http://people.freebsd.org/~jhb/papers/bsdcan/2008/article/node4.html http://freebsd.1045724.n5.nabble.com/Per-core-per-device-interrupt-counts-td4050285.html

2013-11-19 23:26 Updated by: yuichi_xy
Comment

FreeBSDの起動メッセージの差分を比較したところ、次のような違いがありました。

CPU1個

INTR: Adding local APIC 0 as a target
Event timer "LAPIC" quality 600
ACPI APIC Table: <BOCHS  BXPCAPIC>
lapic6: CMCI unmasked
APIC: CPU 0 has ACPI ID 0

CPU2個

INTR: Adding local APIC 0 as a target
Event timer "LAPIC" quality 600
ACPI APIC Table: <BOCHS  BXPCAPIC>
Reset cpu 0x6
INTR: Adding local APIC 4 as a target
INTR: Adding local APIC 6 as a target
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
 cpu0 (BSP): APIC ID:  4
 cpu1 (AP): APIC ID:  6
lapic4: CMCI unmasked
APIC: CPU 0 has ACPI ID 0
APIC: CPU 1 has ACPI ID 1

2013-11-20 21:30 Updated by: yuichi_xy
Comment

usr/src/sys/x86/x86/intr_machdep.c

/*                                                                              
 * Add a CPU to our mask of valid CPUs that can be destinations of              
 * interrupts.                                                                  
 */
void
intr_add_cpu(u_int cpu)
{

        if (cpu >= MAXCPU)
                panic("%s: Invalid CPU ID", __func__);
        if (bootverbose)
                printf("INTR: Adding local APIC %d as a target\n",
                    cpu_apic_ids[cpu]);

        CPU_SET(cpu, &intr_cpus);
}
2013-11-21 23:03 Updated by: yuichi_xy
Comment

次の内容の /etc/rc.local を作成し、起動直後に割り込みの発生数をシリアルコンソールに出力してみました。

  1. #!/bin/sh
  2. vmstat -i > /dev/ttyu0

TinyVisorなしでFreeBSDを起動

interrupt                          total       rate
irq1: atkbd0                           6          0
irq16: ehci0                          51          3
irq23: ehci1                         117          8
cpu0:timer                          3419        244
irq264: xhci0                        119          8
irq265: hdac0                         78          5
irq266: re0                            2          0
irq267: ahci0                        823         58
irq268: ahci1                        123          8
cpu1:timer                           461         32
cpu2:timer                           343         24
cpu3:timer                           378         27
Total                               5920        422

VM1にCPUを2個割り当てFreeBSDを起動

interrupt                          total       rate
irq4: uart0                            1          0
cpu0:timer                         70595       8824
irq264:                                1          0
irq265: re0                            6          0
irq266: ahci0                       1314        164
cpu1:timer                         54226       6778
Total                             126143      15767

VM1にCPUを1個割り当てFreeBSDを起動 (NIC(re0)の割り込みが1回だけだが発生していました)

interrupt                          total       rate
irq4: uart0                            1          0
cpu0:timer                          6165         43
irq265: re0                            1          0
irq266: ahci0                        888          6
Total                               7055         49
2013-11-22 21:31 Updated by: yuichi_xy
Comment

VM1にCPUを1個割り当てて、Linuxを起動し、lspci -vvを実行してみた。

FreeBSDの場合、NICはMSI-Xだが、Linuxの場合、NICもMSIだった。

また、Free BSDの場合Message Dataのビット14(Level)が0だったが、Linuxの場合1だった。しかし、ビット15(Trigger Mode)で0(Edgeモード)を指定しているので、Levelは関係ないはず。

05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI E
xpress Gigabit Ethernet controller (rev 06)
        Subsystem: ASRock Incorporation Motherboard (one of many)
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Step
ping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- 
<MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 40
        Region 0: I/O ports at d000 [size=256]
        Region 2: Memory at f0004000 (64-bit, prefetchable) [size=4K]
        Region 4: Memory at f0000000 (64-bit, prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3h
ot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
                Address: 00000000fee06000  Data: 4029
        Capabilities: [70] Express (v2) Endpoint, MSI 01
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L
1 <64us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupporte
d-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 4096 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPe
nd-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L
0 unlimited, L1 <64us
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [b0] MSI-X: Enable- Count=4 Masked-
                Vector table: BAR=4 offset=00000000
                PBA: BAR=4 offset=00000800
        Capabilities: [d0] Vital Product Data
                No end tag found
        Kernel driver in use: r8169

06:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) (prog-if 01 [AHCI 1.0])
        Subsystem: ASRock Incorporation Device 0612
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 41
        Region 0: I/O ports at c050 [size=8]
        Region 1: I/O ports at c040 [size=4]
        Region 2: I/O ports at c030 [size=8]
        Region 3: I/O ports at c020 [size=4]
        Region 4: I/O ports at c000 [size=32]
        Region 5: Memory at f7800000 (32-bit, non-prefetchable) [size=512]
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
                Address: fee06000  Data: 4041
        Capabilities: [78] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [80] Express (v2) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s, Latency L0 <512ns, L1 <2us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABC, TimeoutDis+
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Kernel driver in use: ahci
2013-11-22 22:22 Updated by: yuichi_xy
  • Details Updated
Comment

yuichi_xy への返信

FreeBSDの場合、NICはMSI-Xだが、Linuxの場合、NICもMSIだった。

more /boot/loader.conf.local に以下の行を追加し、MSI-Xを無効化し、MSIを有効化してみた。

hw.re.msix_disable=1

それでも、NICがリンクアップしなかった。

re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F PCIe Gigabit Ethernet> port 0xd000-0xd0f
f mem 0xf0004000-0xf0004fff,0xf0000000-0xf0003fff at device 0.0 on pci5
re0: MSI count : 1
re0: MSI-X count : 4
re0: attempting to allocate 1 MSI vectors (1 supported)
msi: routing MSI IRQ 265 to local APIC 6 vector 59
re0: using IRQ 265 for MSI
re0: Using 1 MSI message
re0: Chip rev. 0x2c800000
re0: MAC rev. 0x00000000
miibus0: <MII bus> on re0
rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus0
rgephy0: OUI 0x00e04c, model 0x0011, rev. 5
rgephy0:  none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX
, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-
master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
re0: bpf attached
re0: Ethernet address: bc:5f:f4:97:58:96

...

start_init: trying /sbin/init
re0: link state changed to DOWN

FreeBSD/amd64 (xxxxx) (ttyu0)

login:
2013-11-23 20:17 Updated by: yuichi_xy
Comment

if_re.cに定義されている他のパラメータも試してみたが、NICがリンクアップしない事象が発生した。

hw.re.intr_filter=1
hw.re.prefer_iomap=1
2013-12-31 22:43 Updated by: yuichi_xy
Comment

r144でも再現しました。

2014-05-04 18:13 Updated by: yuichi_xy
Comment

r193とFreeBSD 10.0の組み合わせでも発生しました。

2014-09-20 20:26 Updated by: yuichi_xy
Comment

r279とFreeBSD 10.0の組み合わせでも再現しました。

2015-09-23 23:30 Updated by: yuichi_xy
Comment

r346とFreeBSD 10.2の組み合わせでも再現しました。

2016-07-24 20:21 Updated by: yuichi_xy
Comment

25b1818336f9 と FreeBSD 10.2の組み合わせでは、問題は発生しませんでした。

つまり、VM1にCPUを1個しか割り当てずにVM1でFreeBSDを起動しても、NICがリンクアップしました。

2016-07-24 21:52 Updated by: yuichi_xy
  • Resolution Update from None to Works For Me
  • Ticket Close date is changed to 2016-07-24 21:52
  • Status Update from Open to Closed
Comment

yuichi_xy への返信

25b1818336f9 と FreeBSD 10.2の組み合わせでは、問題は発生しませんでした。 つまり、VM1にCPUを1個しか割り当てずにVM1でFreeBSDを起動しても、NICがリンクアップしました。

release 1.7 (84802a6df98c)と FreeBSD 10.2の組み合わせでも、問題は発生しませんでした。 いつの間にか発生しなくなってしまったようです。

チケットの状態は完了にします。

Attachment File List

Edit

You are not logged in. I you are not logged in, your comment will be treated as an anonymous post. » Login