Uploaded image for project: 'Near Realtime RAN Intelligent Controller'
  1. Near Realtime RAN Intelligent Controller
  2. RIC-1042

rmrSendToE2T fails in submgr to E2T (WAS: Can't register xApp)

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Medium Medium
    • None
    • None
    • ric-dep, submgr
    • None

      I am currently following the documentation from https://docs.o-ran-sc.org/projects/o-ran-sc-ric-plt-ric-dep/en/latest/installation-guides.html#installing-near-realtime-ric-in-ric-cluster.
      I am currently using the KPIMON xApp to test everything.

      That's where the problems start... The app doesn't seem to register properly:

      {"ts":1705503041178,"crit":"DEBUG","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"7","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-17T14:50:41"},"msg":"Application='kpimon-go' is not ready yet, waiting ..."}
      {"ts":1705503042160,"crit":"DEBUG","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"7","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-17T14:50:42"},"msg":"App registration is not done yet, sleep 5s and check again"}

      I then tried to register the xApp manually:

      curl -X 'POST' 'http://10.110.43.166:8080/ric/v1/register' -H 'accept: application/json' -H 'Content-Type: application/json' -d '{
        "appName": "kpimon-go",
        "appVersion": "2.0.1",
        "appInstanceName": "kpimon-go",
        "httpEndpoint": "10.100.236.127:8080",
        "rmrEndpoint": "10.97.0.236:4560",
        "config": ""
      }'

      Unfortunately, I don't know exactly how to validate whether this works successfully.
      When I output the logs from appmgr, I only see the logs that are output directly after the start. Afterwards the logs are empty:

      cp: cannot stat '/opt/ric/certificates/*': No such file or directory
      2024/01/17 14:03:47 Using config file: /opt/ric/config/appmgr.yaml
      redis: got 7 elements in COMMAND reply, wanted 6
      2024/01/17 14:03:47 Serving app manager at http://[::]:8080

      What am I missing?

       

      UPDATE:

      Now with the help of tchyang I got a step further.
      Unfortunately I still have problems..

      It seems that the rmrSendToE2T fails during a new CRESTSubscriptionRequest.

      Here is the logging I get when trying to get the KPIMON-GO xApp up and running.

       
      submgr:

      CRESTSubscriptionRequest SubscriptionID='' ClientEndpoint.Host=service-ricxapp-kpimon-go-http.ricxapp ClientEndpoint.HTTPPort=8080 ClientEndpoint.RMRPort=4560 Meid=gnb_734_373_16b8cef1 E2SubscriptionDirectives=nil RANFunctionID=2 SubscriptionDetail.XappEventInstanceID=1 SubscriptionDetail.EventTriggers= [83915] SubscriptionDetail.ActionToBeSetup.ActionID=1 SubscriptionDetail.ActionToBeSetup.ActionType=report SubscriptionDetail.ActionToBeSetup.ActionDefinition= [] SubscriptionDetail.ActionToBeSetup.SubsequentAction.SubsequentActionType=continue SubscriptionDetail.ActionToBeSetup..SubsequentAction.TimeToWait=w10ms 
      {"ts":1706002516827,"crit":"ERROR","id":"submgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:16"},"msg":"rmrSendToE2T: Send failed: Failed with retries(10) params(Src= Mtype=12010 SubId=3 Xid= Meid=meid(RanName=gnb_734_373_16b8cef1) Paylens=42/42 Paymd5=d7d09a18b0a345831845933b703cbc36)"} {"ts":1706002516827,"crit":"ERROR","id":"submgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:16"},"msg":"rmrSendToE2T() failed:Failed with retries(10) params(Src= Mtype=12010 SubId=3 Xid= Meid=meid(RanName=gnb_734_373_16b8cef1) Paylens=42/42 Paymd5=d7d09a18b0a345831845933b703cbc36)"} {"ts":1706002523835,"crit":"ERROR","id":"submgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:23"},"msg":"rmrSendToE2T: Send failed: Failed with retries(10) params(Src= Mtype=12010 SubId=3 Xid= Meid=meid(RanName=gnb_734_373_16b8cef1) Paylens=42/42 Paymd5=d7d09a18b0a345831845933b703cbc36)"} {"ts":1706002523835,"crit":"ERROR","id":"submgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:23"},"msg":"rmrSendToE2T() failed:Failed with retries(10) params(Src= Mtype=12010 SubId=3 Xid= Meid=meid(RanName=gnb_734_373_16b8cef1) Paylens=42/42 Paymd5=d7d09a18b0a345831845933b703cbc36)"} {"ts":1706002530843,"crit":"ERROR","id":"submgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:30"},"msg":"rmrSendToE2T: Send failed: Failed with retries(10) params(Src= Mtype=12020 SubId=3 Xid= Meid=meid(RanName=gnb_734_373_16b8cef1) Paylens=22/22 Paymd5=66cd95567f4177ca7ec5ebe4cd90a1f3)"} {"ts":1706002530843,"crit":"ERROR","id":"submgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:30"},"msg":"SUBS-SubDelReq: rmrSendToE2T failure: transsubs(trans(5/meid(RanName=gnb_734_373_16b8cef1))) subs(reqid(1:3)/meid(RanName=gnb_734_373_16b8cef1)/service-ricxapp-kpimon-go-rmr.ricxapp:4560) transxapp(trans(4/meid(RanName=gnb_734_373_16b8cef1))/transkey(service-ricxapp-kpimon-go-rmr.ricxapp:4560/2bLm0d2xSR5wPylCMUPupD29XVD)/0) err(Failed with retries(10) params(Src= Mtype=12020 SubId=3 Xid= Meid=meid(RanName=gnb_734_373_16b8cef1) Paylens=22/22 Paymd5=66cd95567f4177ca7ec5ebe4cd90a1f3))"} {"ts":1706002537852,"crit":"ERROR","id":"submgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:37"},"msg":"rmrSendToE2T: Send failed: Failed with retries(10) params(Src= Mtype=12020 SubId=3 Xid= Meid=meid(RanName=gnb_734_373_16b8cef1) Paylens=22/22 Paymd5=66cd95567f4177ca7ec5ebe4cd90a1f3)"} {"ts":1706002537852,"crit":"ERROR","id":"submgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:37"},"msg":"SUBS-SubDelReq: rmrSendToE2T failure: transsubs(trans(5/meid(RanName=gnb_734_373_16b8cef1))) subs(reqid(1:3)/meid(RanName=gnb_734_373_16b8cef1)/service-ricxapp-kpimon-go-rmr.ricxapp:4560) transxapp(trans(4/meid(RanName=gnb_734_373_16b8cef1))/transkey(service-ricxapp-kpimon-go-rmr.ricxapp:4560/2bLm0d2xSR5wPylCMUPupD29XVD)/0) err(Failed with retries(10) params(Src= Mtype=12020 SubId=3 Xid= Meid=meid(RanName=gnb_734_373_16b8cef1) Paylens=22/22 Paymd5=66cd95567f4177ca7ec5ebe4cd90a1f3))"} {"ts":1706002539855,"crit":"ERROR","id":"submgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:39"},"msg":"XAPP-SubReq E2 subscription failed: transxapp(trans(4/meid(RanName=gnb_734_373_16b8cef1))/transkey(service-ricxapp-kpimon-go-rmr.ricxapp:4560/2bLm0d2xSR5wPylCMUPupD29XVD)/0) subs(reqid(1:3)/meid(RanName=gnb_734_373_16b8cef1)/) err(E2 RICSubscriptionResponse timeout)"}

      Listing of current subscriptions for that E2 node: 

      {"2bLm0d2xSR5wPylCMUPupD29XVD":{"Created":"2024-01-23 09:35:11.816","Meid":"gnb_734_373_16b8cef1","InstanceIds":null,"SubReqOngoing":false,"SubDelReqOngoing":false}}

      kpimon-go xAPP:

      RMR is ready now ...
      {"ts":1706002510772,"crit":"DEBUG","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:10"},"msg":"Application='kpimon-go' is now up and ready, continue with registration ..."} {"ts":1706002510772,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:10"},"msg":"getService: SERVICE_RICXAPP_KPIMON-GO_HTTP_PORT [tcp: 10.110.120.22:8080]"} {"ts":1706002510772,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:10"},"msg":"getService: SERVICE_RICXAPP_KPIMON-GO_RMR_PORT [tcp: 10.103.16.20:4560]"} {"ts":1706002510774,"crit":"DEBUG","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:10"},"msg":"restapi: method=GET url=/ric/v1/config"} {"ts":1706002510774,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:10"},"msg":"Inside appconfigHandler"} {"ts":1706002510775,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:10"},"msg":"Post to 'http://service-ricplt-appmgr-http.ricplt:8080/ric/v1/register' done, status:201 Created"} {"ts":1706002510775,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:10"},"msg":"Registration done, proceeding with startup ..."} {"ts":1706002510823,"crit":"DEBUG","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:10"},"msg":"restapi: method=GET url=/ric/v1/config"} {"ts":1706002510823,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:10"},"msg":"Inside appconfigHandler"} {"ts":1706002511813,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:11"},"msg":"App registration is done, ready to send subscription request."} {"ts":1706002511813,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:11"},"msg":"Before send subscription request to gnb_734_373_16b8cef1"} {"ts":1706002511813,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:11"},"msg":"Sending subscription request for MEID: gnb_734_373_16b8cef1"} {"ts":1706002511814,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:11"},"msg":"*****body: {\n \"ClientEndpoint\": {\n \"HTTPPort\": 8080,\n \"Host\": \"service-ricxapp-kpimon-go-http.ricxapp\",\n \"RMRPort\": 4560\n },\n \"Meid\": \"gnb_734_373_16b8cef1\",\n \"RANFunctionID\": 2,\n \"SubscriptionDetails\": [\n {\n \"ActionToBeSetupList\": [\n {\n \"ActionID\": 1,\n \"ActionType\": \"report\",\n \"SubsequentAction\": {\n \"SubsequentActionType\": \"continue\",\n \"TimeToWait\": \"w10ms\"\n }\n }\n ],\n \"EventTriggers\": [\n 8,\n 39,\n 15\n ],\n \"XappEventInstanceId\": 1\n }\n ]\n}"} {"ts":1706002511817,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:11"},"msg":"Successfully subscription done (gnb_734_373_16b8cef1), subscriptrion id: 2bLm0d2xSR5wPylCMUPupD29XVD"} {"ts":1706002511817,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:11"},"msg":"After send subscription request to gnb_734_373_16b8cef1"} len of ranUeKpi= 1 ranUeKpi map = map[gnb_734_373_16b8cef1:[ DRB.RlcSduTransmittedVolumeDL_Filter DRB.RlcSduTransmittedVolumeUL_Filter DRB.PerDataVolumeDLDist.Bin DRB.PerDataVolumeULDist.Bin DRB.RlcPacketDropRateDLDist DRB.PacketLossRateULDist L1M.DL-SS-RSRP.SSB L1M.DL-SS-SINR.SSB L1M.UL-SRS-RSRP]] len of ranCellKpi= 1 ranCellKpi map = map[gnb_734_373_16b8cef1:[ DRB.RlcSduTransmittedVolumeDL_Filter DRB.RlcSduTransmittedVolumeUL_Filter DRB.PerDataVolumeDLDist.Bin DRB.PerDataVolumeULDist.Bin DRB.RlcPacketDropRateDLDist DRB.PacketLossRateULDist L1M.DL-SS-RSRP.SSB L1M.DL-SS-SINR.SSB L1M.UL-SRS-RSRP]] len of Glob_cell_Plmn= 0 Glob_cell_Plmn map = map[] {"ts":1706002511817,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:11"},"msg":"End callback KPI monitor xApp ..."} {"ts":1706002521057,"crit":"DEBUG","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:21"},"msg":"restapi: method=GET url=/ric/v1/health/alive"} {"ts":1706002522068,"crit":"DEBUG","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"8","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2024-01-23T09:35:22"},"msg":"restapi: method=GET url=/ric/v1/health/ready"}

       

       

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            singh.sunil SUNIL SINGH
            fklement Felix Klement
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: