5 Replies Latest reply on Sep 4, 2013 10:22 AM by swatt

    After Upgrade to 9, Agents don't respond or run agent tasks

    swatt Rookie

      I upgraded to version 9 last week - Aug 28.  Didn't do much else at the time except poke around the console.

       

      Today I ran it up again to follow up on the last patch cycle.  We use agent-based deployment with patch groups.  However, anything I try to get the agents to do, doesn't work.  The response is an almost immediate "Agent didn't respond"

       

      Agent commands I've tried:

           Agents / Run task from policy

           Agents / Clear retry counts

       

      Last Agent Check-in dates are current.  Push-based scanning and patching works.

       

      However, Last Patch Scan dates, scheduled for daily in the policies, are exactly 1 week old - from Aug 28.  In the agent GUI, it *did* run the scheduled patch scan today - Sept 4 - and the GUI says that results were sent, but the console doesn't seem to be receiving these results.  So the agents can't be controlled, AND, they stopped sending results to the console since the upgrade.

       

      The STAgent.log file on the agent contains the reason:

      -----

      2013-09-04T12:55:53.5899736Z 2244 E QueueManager.cpp:276 File transfer failed with error: class STWin32::CComException at ResultImportClient.cpp:89: Unable to send results to any host.: The remote endpoint is unable to process the request due to being overloaded..

      2013-09-04T12:58:58.6156330Z 1b18 E ResultImportClient.cpp:75 Could not send results to host 'https://TMRMNG07:3121': error: 0x803d0012, client error: 'There was an error communicating with the endpoint at 'https://TMRMNG07:3121/ST/Console/ResultImport/Importer'. ; The server returned HTTP status code '503 (0x1F7)' with text 'Service Unavailable'. ; The service is temporarily overloaded. ; '.

      2013-09-04T12:58:58.6273522Z 1b18 E ResultImportClient.cpp:75 Could not send results to host 'https://tmrmng07:3121/': error: 0x803d0012, client error: 'There was an error communicating with the endpoint at 'https://tmrmng07:3121/ST/Console/ResultImport/Importer'. ; The server returned HTTP status code '503 (0x1F7)' with text 'Service Unavailable'. ; The service is temporarily overloaded. ; '.

      2013-09-04T12:58:58.6410246Z 1b18 E ResultImportClient.cpp:75 Could not send results to host 'https://tmrmng07.tokiomillennium.com:3121/': error: 0x803d0012, client error: 'There was an error communicating with the endpoint at 'https://tmrmng07.tokiomillennium.com:3121/ST/Console/ResultImport/Importer'. ; The server returned HTTP status code '503 (0x1F7)' with text 'Service Unavailable'. ; The service is temporarily overloaded. ; '.

      2013-09-04T12:58:58.6507906Z 1b18 E ResultImportClient.cpp:75 Could not send results to host 'https://192.168.60.56:3121/': error: 0x803d0012, client error: 'There was an error communicating with the endpoint at 'https://192.168.60.56:3121/ST/Console/ResultImport/Importer'. ; The server returned HTTP status code '503 (0x1F7)' with text 'Service Unavailable'. ; The service is temporarily overloaded. ; '.

      2013-09-04T12:58:58.6517672Z 1b18 E QueueManager.cpp:276 File transfer failed with error: class STWin32::CComException at ResultImportClient.cpp:89: Unable to send results to any host.: The remote endpoint is unable to process the request due to being overloaded..

      -----

       

      Please help. It seems very unlikely that the server has suddenly become too overloaded to process agent data, starting exactly on the day of the upgrade.  But hopefully this error information can help lead to the true resolution.

       

      Thanks,

      Sandra

        • 1. Re: After Upgrade to 9, Agents don't respond or run agent tasks
          Rookie

          Sandra,

           

          a wild guess, but have you tried to reinstall the agent with policy?

          After upgrading I had to reinstall all my agents but YMMV.

           

          regards

          Patrick

          • 2. Re: After Upgrade to 9, Agents don't respond or run agent tasks
            SupportEmployee

            Hi Sandra,

             

            I would suggest a reboot of the Protect console system, or if that's not possible at least try closing Protect, stop and re-start the Shavlik Protect Console service. It might be a good idea to run database maintenance as well. (Tools > database maintenance).

             

            Try forcing a check in with all your agents after this. The agents should automatically upgrade to a version 9 agent once they check in. Otherwise, they may still be running an agent of the previous version.

             

            I hope that helps. If this doesn't help, I would suggest opening a case directly with support and provide logs for both console and agent as described in the following document:

            http://community.shavlik.com/docs/DOC-22921

             

            Support contact info here:

            http://www.shavlik.com/support/contact/

            1 of 1 people found this helpful
            • 3. Re: After Upgrade to 9, Agents don't respond or run agent tasks
              swatt Rookie

              Ok, thank you both for the responses. 

               

              I first tried a reinstall of an agent with the same policy it had before.  It installed and registered successfully, and now showed with v9 in the console.  I then asked it to perform a patch scan, which is a task in its agent policy.  It didn't fail to respond, but in the consoled showed as 'running scan'.  In the agent GUI, it was actually caught in a loop doing repeated check-ins every minute or so.  I read some email, and came back to check if it's still doing check-ins, only to find that the agent was no longer installed.  That was wierd, and I decided to move on to the other suggestion.

               

              Since I could reboot the server, I did.  I see now that all the agents are showing v8.  I tried to issue a check-in request to one of the v8 agents, but it doesn't work - Agent didn't respond. 

               

              Back to the one agent that I reinstalled earlier.  I installed it again from the console, and it installed v9 again.  This time there is no check-in loop, but it also doesn't seem to be healthy.  The agent GUI, under System, shows:

              • Checking latest program versions
              • Update failed: Unable to update.  The manifest with platform version '8.0.xxxxx' <sorry, didn't write down the rest, but basically it doesn't match up with the console which is '9.0.xxxxx'>

              And the Patch tab shows the agent tasks, but everything is greyed out.

               

              This led me to do a sync of Core data files to the distribution server.  After that, I installed the same agent again, but this time, no message about Update failed.  And from that point on, it can receive all console commands, and is able to run tasks from the agent policy.

               

              So it would appear the root cause was that Core data files did not automatically sync to the distribution server.  This leaves me with a couple of remaining questions, if you please.

              • Shouldn't Core data files be syncing on a schedule?  I'm pretty sure I had that set up before.  Perhaps the upgrade removed it?
              • Given that I can't send out agent check-in commands to the v8 agents, will they automatically check in and get updated on their own?

               

              Thanks again for the quick responses!

              Sandra

              • 4. Re: After Upgrade to 9, Agents don't respond or run agent tasks
                SupportEmployee
                • Shouldn't Core data files be syncing on a schedule?  I'm pretty sure I had that set up before.  Perhaps the upgrade removed it?

                  Yes it should, but that's my guess as well...the automatic download and syncs might need to be set up again after the upgrade.

                • Given that I can't send out agent check-in commands to the v8 agents, will they automatically check in and get updated on their own?

                  The agents will automatically attempt check in, and they should upgrade after their initial check in after the upgrade. If they for some reason can't check in, you may need to reinstall the agent or check that all requirements are still being met (ports, etc.).

                 

                Hope that helps.

                • 5. Re: After Upgrade to 9, Agents don't respond or run agent tasks
                  swatt Rookie

                  Ok great.  It sounds like the agents will resolve themselves given some time.  I'll check up on all the download and sync schedules.  Thanks for all the helpful suggestions.  I hope you won't mind if I mark my last response as the Correct answer, since it is the one that states the root cause.  All others will be Helpful answers.  If I don't post back, you can assume that the remaining agents sorted themselves out and that all's well.  Thanks again!

                   

                  Edit: Sorry, I can only mark one as Helpful.