Uploaded image for project: 'Near Realtime RAN Intelligent Controller'
  1. Near Realtime RAN Intelligent Controller
  2. RIC-872

RMR routing statistic data printout crash

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: High High
    • F
    • None
    • rmr
    • None

      RMR version "ric message routing library on SI95 p=4560 mv=3 flg=00 id=a (d5197f5 4.8.0 built: Dec 2 2021"

      rmr message sending triggers the routing entry statistics printout every ~5 minutes. When traversing entries for statistics print the mutex lock is missing . So if the routing update happens in same time, then this will cause the crash. Following is the trace (incomplete printout when printing the entry statistics):

      1640562413268 1/RMR [INFO] sends: ts=1640562413 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=172.30.202.103:38000 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562413268 1/RMR [INFO] sends: ts=1640562413 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricxapp-dddd-rmr.ricxapp:4560 open=1 succ=215531 fail=0 (hard=0 soft=0)
      1640562413268 1/RMR [INFO] sends: ts=1640562413 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricxapp-dddd-rmr.ricxapp:4560 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562413268 1/RMR [INFO] sends: ts=1640562413 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricplt-submgr-rmr.ricplt:4560 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562413268 1/RMR [INFO] sends: ts=1640562413 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricxapp-ddddd-rmr.ricxapp:4560 open=1 succ=6055874824 fail=149 (hard=6 soft=143)
      1640562413268 1/RMR [INFO] sends: ts=1640562413 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricplt-e2mgr-rmr.ricplt:3801 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562413268 1/RMR [INFO] sends: ts=1640562413 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricxapp-ueec-rmr.ricxapp:4560 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562413268 1/RMR [INFO] sends: ts=1640562413 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricplt-a1mediator-rmr.ricplt:4562 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562714290 1/RMR [INFO] sends: ts=1640562714 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricxapp-dddd-rmr.ricxapp:4560 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562714290 1/RMR [INFO] sends: ts=1640562714 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricxapp-ddddd-rmr.ricxapp:4560 open=1 succ=6228540660 fail=0 (hard=0 soft=0)
      1640562714290 1/RMR [INFO] sends: ts=1640562714 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=172.30.202.103:38000 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562714290 1/RMR [INFO] sends: ts=1640562714 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricxapp-ddddd-rmr.ricxapp:4560 open=1 succ=215832 fail=0 (hard=0 soft=0)
      1640562714290 1/RMR [INFO] sends: ts=1640562714 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricxapp-dddddd-rmr.ricxapp:4560 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562714290 1/RMR [INFO] sends: ts=1640562714 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricplt-submgr-rmr.ricplt:4560 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562714290 1/RMR [INFO] sends: ts=1640562714 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricxapp-dddddd-rmr.ricxapp:4560 open=1 succ=6064529241 fail=149 (hard=6 soft=143)
      1640562714290 1/RMR [INFO] sends: ts=1640562714 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricplt-e2mgr-rmr.ricplt:3801 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562714290 1/RMR [INFO] sends: ts=1640562714 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricxapp-ueec-rmr.ricxapp:4560 open=0 succ=0 fail=0 (hard=0 soft=0)
      1640562714290 1/RMR [INFO] sends: ts=1640562714 src=service-ricxapp-dddterm-rmr.ricxapp:4560 target=service-ricplt-a1mediator-rmr.ricplt:4562 open=0 succ=0 fail=0 (hard=0 soft=0)
      fatal error: unexpected signal during runtime execution
      [signal SIGSEGV: segmentation violation code=0x1 addr=0x1a948 pc=0x7f9874a5a0be]

      runtime stack:
      runtime.throw(0x10d7765, 0x2a)
      /opt/go/1.16/src/runtime/panic.go:1117 +0x72
      runtime.sigpanic()
      /opt/go/1.16/src/runtime/signal_unix.go:718 +0x2e5

      goroutine 776 [syscall]:
      runtime.cgocall(0xd6312b, 0xc006251e18, 0x7f970800ade0)
      /opt/go/1.16/src/runtime/cgocall.go:154 +0x5b fp=0xc006251de8 sp=0xc006251db0 pc=0x40703b
      gerrit.o-ran-sc.org/r/ric-plt/xapp-frame/pkg/xapp._Cfunc_rmr_send_msg(0x7f9820000d10, 0x7f9784368f90, 0x0)
      _cgo_gotypes.go:420 +0x49 fp=0xc006251e18 sp=0xc006251de8 pc=0xaf8d69
      gerrit.o-ran-sc.org/r/ric-plt/xapp-frame/pkg/xapp.(*RMRClient).SendBuf.func3(0xc00020ca80, 0x7f9784368f90, 0xc006251ee8)
      /ws/cache/gocache/pkg/mod/gerrit.o-ran-sc.org/r/ric-plt/xapp-frame.git@v0.9.8/pkg/xapp/rmr.go:430 +0x86 fp=0xc006251e50 sp=0xc006251e18 pc=0xafe3e6
      gerrit.o-ran-sc.org/r/ric-plt/xapp-frame/pkg/xapp.(*RMRClient).SendBuf(0xc00020ca80, 0x7f9784368f90, 0x7f9784368f00, 0x0, 0x0)
      /ws/cache/gocache/pkg/mod/gerrit.o-ran-sc.org/r/ric-plt/xapp-frame.git@v0.9.8/pkg/xapp/rmr.go:430 +0x3eb fp=0xc006251ec0 sp=0xc006251e50 pc=0xafb74b
      gerrit.o-ran-sc.org/r/ric-plt/xapp-frame/pkg/xapp.(*RMRClient).Send(0xc00020ca80, 0xc004962f80, 0xe4b900, 0xc05c831ac0)
      /ws/cache/gocache/pkg/mod/gerrit.o-ran-sc.org/r/ric-plt/xapp-frame.git@v0.9.8/pkg/xapp/rmr.go:410 +0x6f fp=0xc006251ef8 sp=0xc006251ec0 pc=0xafb30f
      main.(*dddTerm).ConverterSend(0xc000562460, 0xc004962f80, 0x16)
      /ws/cmd/dddterm/dddterm.go:232 +0x3e fp=0xc006251f28 sp=0xc006251ef8 pc=0xd58c3e
      main.(*pipelineSendGeo).Run(0xc09ac6caa0)
      /ws/cmd/dddterm/conv_geo.go:59 +0x1c2 fp=0xc006251f80 sp=0xc006251f28 pc=0xd531c2
      ddddd.com/ric/dddddpkgs/pkg/worker.(*WorkerQueue).Handle(0xc011ace240, 0x0)
      /ws/cache/gocache/pkg/mod/bhgerrit.ext.net.nokia.com/ric/nokiapkgs.git@v1.7.0/pkg/worker/workerqueue.go:87 +0xd9 fp=0xc006251fd0 sp=0xc006251f80 pc=0xb7c179
      runtime.goexit()
      /opt/go/1.16/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc006251fd8 sp=0xc006251fd0 pc=0x470d81
      created by ddddddd.com/ric/nokiapkgs/pkg/worker.(*WorkerQueue).ResizeRunners
      /ws/cache/gocache/pkg/mod/ddddddd.com/ric/ddddd.git@v1.7.0/pkg/worker/workerqueue.go:109 +0x7f

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            alexandrehuff Alexandre Huff
            ehietala Erkki Hietala
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: