archive-org.com » ORG » Z » ZENOSS.ORG

Total: 955

Choose link from "Titles, links and description words view":

Or switch to "Titles and links view".
  • Zenoss Community - Open Source Network Monitoring and Systems Management
    1 7 3 Submitted by Jcurry on Mon 01 04 2016 12 43 Forums ZenPack Development Zenoss Core 4 2 5 SUP 457 I am writing a ZenPack that uses the PythonCollector 1 7 3 and creates a dsplugin in fact 3 dsplugins I have 2 test devices very similar both have the same templates bound that use these dsplugins one is a device template 2 are component Email jane

    Original URL path: http://www.zenoss.org/tags/pythoncollector-173 (2016-01-08)
    Open archived version from archive


  • Zenoss Community - Open Source Network Monitoring and Systems Management
    of any PythonDataSourcePlugin is called The timer is cancelled as soon as that collect method returns If the timer reaches blockingtimeout before the collect method returns the zenpython process completely restarts itself with the plugin that failed to return in time disabled This may sound extreme but for technical reasons was really the only way to recover from being blocked To understand why zenpython must restart itself you have to understand how zenpython is a single threaded process that uses asynchronous IO via the Twisted library to efficiently perform lots of parallel collection Anytime non asynchronous code is being executed it means that no other collection or process can be performed until it yields back to the reactor Twisted s event loop The catch here is that zenpython is executing arbitrary code provided by ZenPacks The zenpython code essentially gives up control to that ZenPack code as soon as it calls it until that code returns The way that zenpython manages to restart itself when it calls to ZenPack code that doesn t return is by running a watchdog thread that keeps an eye on that timer and restarts the process if it expires So another question you d ask and I think you implied is why disable the plugin for 2 devices when it only timed out on 1 device To understand why that is done you have to imagine a scenario where you have a plugin that executes on all Linux servers Let s say you have 1 000 Linux servers being monitored and let s say that 50 of them develop some condition that causes the plugin to block on them for 10 seconds Assuming it takes zenpython only 1 minute to restart when monitoring 1 000 servers it would take 60 seconds 10 seconds 50 servers or 11 minutes before zenpython would have all of the plugin device executions disabled During this time zenpython wouldn t be doing anything So we re forced to assume that if a plugin blocks once that it s not written correctly and using blocking IO So the plugin is completely disabled for all devices that use it So now back to why the default blockingtimeout is set to 5 which may seem way too low given everything above To understand this you again have to understand why the PythonDataSourcePlugin collect method must return a Deferred After all it s easy to imagine making an API call to some application that takes more than 5 seconds When using Twisted s non blocking networking APIs you immediately get a Deferred object returned when you make a call like getPage http example com api really slow thing even if the response doesn t come back for 10 minutes When the response comes back the Deferred s callback or errback if there s an error is called The PythonDataSourcePlugin collect must do the same thing Usually by returning the Deferred object that something like getPage http returned to you If you properly use Twisted

    Original URL path: http://www.zenoss.org/comment/22836 (2016-01-08)
    Open archived version from archive

  • Zenoss Community - Open Source Network Monitoring and Systems Management
    of any PythonDataSourcePlugin is called The timer is cancelled as soon as that collect method returns If the timer reaches blockingtimeout before the collect method returns the zenpython process completely restarts itself with the plugin that failed to return in time disabled This may sound extreme but for technical reasons was really the only way to recover from being blocked To understand why zenpython must restart itself you have to understand how zenpython is a single threaded process that uses asynchronous IO via the Twisted library to efficiently perform lots of parallel collection Anytime non asynchronous code is being executed it means that no other collection or process can be performed until it yields back to the reactor Twisted s event loop The catch here is that zenpython is executing arbitrary code provided by ZenPacks The zenpython code essentially gives up control to that ZenPack code as soon as it calls it until that code returns The way that zenpython manages to restart itself when it calls to ZenPack code that doesn t return is by running a watchdog thread that keeps an eye on that timer and restarts the process if it expires So another question you d ask and I think you implied is why disable the plugin for 2 devices when it only timed out on 1 device To understand why that is done you have to imagine a scenario where you have a plugin that executes on all Linux servers Let s say you have 1 000 Linux servers being monitored and let s say that 50 of them develop some condition that causes the plugin to block on them for 10 seconds Assuming it takes zenpython only 1 minute to restart when monitoring 1 000 servers it would take 60 seconds 10 seconds 50 servers or 11 minutes before zenpython would have all of the plugin device executions disabled During this time zenpython wouldn t be doing anything So we re forced to assume that if a plugin blocks once that it s not written correctly and using blocking IO So the plugin is completely disabled for all devices that use it So now back to why the default blockingtimeout is set to 5 which may seem way too low given everything above To understand this you again have to understand why the PythonDataSourcePlugin collect method must return a Deferred After all it s easy to imagine making an API call to some application that takes more than 5 seconds When using Twisted s non blocking networking APIs you immediately get a Deferred object returned when you make a call like getPage http example com api really slow thing even if the response doesn t come back for 10 minutes When the response comes back the Deferred s callback or errback if there s an error is called The PythonDataSourcePlugin collect must do the same thing Usually by returning the Deferred object that something like getPage http returned to you If you properly use Twisted

    Original URL path: http://www.zenoss.org/comment/22946 (2016-01-08)
    Open archived version from archive

  • Zenoss Community - Open Source Network Monitoring and Systems Management
    of any PythonDataSourcePlugin is called The timer is cancelled as soon as that collect method returns If the timer reaches blockingtimeout before the collect method returns the zenpython process completely restarts itself with the plugin that failed to return in time disabled This may sound extreme but for technical reasons was really the only way to recover from being blocked To understand why zenpython must restart itself you have to understand how zenpython is a single threaded process that uses asynchronous IO via the Twisted library to efficiently perform lots of parallel collection Anytime non asynchronous code is being executed it means that no other collection or process can be performed until it yields back to the reactor Twisted s event loop The catch here is that zenpython is executing arbitrary code provided by ZenPacks The zenpython code essentially gives up control to that ZenPack code as soon as it calls it until that code returns The way that zenpython manages to restart itself when it calls to ZenPack code that doesn t return is by running a watchdog thread that keeps an eye on that timer and restarts the process if it expires So another question you d ask and I think you implied is why disable the plugin for 2 devices when it only timed out on 1 device To understand why that is done you have to imagine a scenario where you have a plugin that executes on all Linux servers Let s say you have 1 000 Linux servers being monitored and let s say that 50 of them develop some condition that causes the plugin to block on them for 10 seconds Assuming it takes zenpython only 1 minute to restart when monitoring 1 000 servers it would take 60 seconds 10 seconds 50 servers or 11 minutes before zenpython would have all of the plugin device executions disabled During this time zenpython wouldn t be doing anything So we re forced to assume that if a plugin blocks once that it s not written correctly and using blocking IO So the plugin is completely disabled for all devices that use it So now back to why the default blockingtimeout is set to 5 which may seem way too low given everything above To understand this you again have to understand why the PythonDataSourcePlugin collect method must return a Deferred After all it s easy to imagine making an API call to some application that takes more than 5 seconds When using Twisted s non blocking networking APIs you immediately get a Deferred object returned when you make a call like getPage http example com api really slow thing even if the response doesn t come back for 10 minutes When the response comes back the Deferred s callback or errback if there s an error is called The PythonDataSourcePlugin collect must do the same thing Usually by returning the Deferred object that something like getPage http returned to you If you properly use Twisted

    Original URL path: http://www.zenoss.org/comment/22951 (2016-01-08)
    Open archived version from archive

  • Zenoss Community - Open Source Network Monitoring and Systems Management
    of any PythonDataSourcePlugin is called The timer is cancelled as soon as that collect method returns If the timer reaches blockingtimeout before the collect method returns the zenpython process completely restarts itself with the plugin that failed to return in time disabled This may sound extreme but for technical reasons was really the only way to recover from being blocked To understand why zenpython must restart itself you have to understand how zenpython is a single threaded process that uses asynchronous IO via the Twisted library to efficiently perform lots of parallel collection Anytime non asynchronous code is being executed it means that no other collection or process can be performed until it yields back to the reactor Twisted s event loop The catch here is that zenpython is executing arbitrary code provided by ZenPacks The zenpython code essentially gives up control to that ZenPack code as soon as it calls it until that code returns The way that zenpython manages to restart itself when it calls to ZenPack code that doesn t return is by running a watchdog thread that keeps an eye on that timer and restarts the process if it expires So another question you d ask and I think you implied is why disable the plugin for 2 devices when it only timed out on 1 device To understand why that is done you have to imagine a scenario where you have a plugin that executes on all Linux servers Let s say you have 1 000 Linux servers being monitored and let s say that 50 of them develop some condition that causes the plugin to block on them for 10 seconds Assuming it takes zenpython only 1 minute to restart when monitoring 1 000 servers it would take 60 seconds 10 seconds 50 servers or 11 minutes before zenpython would have all of the plugin device executions disabled During this time zenpython wouldn t be doing anything So we re forced to assume that if a plugin blocks once that it s not written correctly and using blocking IO So the plugin is completely disabled for all devices that use it So now back to why the default blockingtimeout is set to 5 which may seem way too low given everything above To understand this you again have to understand why the PythonDataSourcePlugin collect method must return a Deferred After all it s easy to imagine making an API call to some application that takes more than 5 seconds When using Twisted s non blocking networking APIs you immediately get a Deferred object returned when you make a call like getPage http example com api really slow thing even if the response doesn t come back for 10 minutes When the response comes back the Deferred s callback or errback if there s an error is called The PythonDataSourcePlugin collect must do the same thing Usually by returning the Deferred object that something like getPage http returned to you If you properly use Twisted

    Original URL path: http://www.zenoss.org/comment/22966 (2016-01-08)
    Open archived version from archive

  • Zenoss Community - Open Source Network Monitoring and Systems Management
    of any PythonDataSourcePlugin is called The timer is cancelled as soon as that collect method returns If the timer reaches blockingtimeout before the collect method returns the zenpython process completely restarts itself with the plugin that failed to return in time disabled This may sound extreme but for technical reasons was really the only way to recover from being blocked To understand why zenpython must restart itself you have to understand how zenpython is a single threaded process that uses asynchronous IO via the Twisted library to efficiently perform lots of parallel collection Anytime non asynchronous code is being executed it means that no other collection or process can be performed until it yields back to the reactor Twisted s event loop The catch here is that zenpython is executing arbitrary code provided by ZenPacks The zenpython code essentially gives up control to that ZenPack code as soon as it calls it until that code returns The way that zenpython manages to restart itself when it calls to ZenPack code that doesn t return is by running a watchdog thread that keeps an eye on that timer and restarts the process if it expires So another question you d ask and I think you implied is why disable the plugin for 2 devices when it only timed out on 1 device To understand why that is done you have to imagine a scenario where you have a plugin that executes on all Linux servers Let s say you have 1 000 Linux servers being monitored and let s say that 50 of them develop some condition that causes the plugin to block on them for 10 seconds Assuming it takes zenpython only 1 minute to restart when monitoring 1 000 servers it would take 60 seconds 10 seconds 50 servers or 11 minutes before zenpython would have all of the plugin device executions disabled During this time zenpython wouldn t be doing anything So we re forced to assume that if a plugin blocks once that it s not written correctly and using blocking IO So the plugin is completely disabled for all devices that use it So now back to why the default blockingtimeout is set to 5 which may seem way too low given everything above To understand this you again have to understand why the PythonDataSourcePlugin collect method must return a Deferred After all it s easy to imagine making an API call to some application that takes more than 5 seconds When using Twisted s non blocking networking APIs you immediately get a Deferred object returned when you make a call like getPage http example com api really slow thing even if the response doesn t come back for 10 minutes When the response comes back the Deferred s callback or errback if there s an error is called The PythonDataSourcePlugin collect must do the same thing Usually by returning the Deferred object that something like getPage http returned to you If you properly use Twisted

    Original URL path: http://www.zenoss.org/forum/136876?destination=node/136876 (2016-01-08)
    Open archived version from archive

  • Zenoss Community - Open Source Network Monitoring and Systems Management
    Search Download WIKI FORUMS BLOG Submit a Bug Archive HELP DOCS FORUMS HELP BLOG s User account Primary tabs Log in active tab Request new password Username Enter your Zenoss username Password Enter the password that accompanies your username Zenoss

    Original URL path: http://www.zenoss.org/user/login?destination=forum/39156 (2016-01-08)
    Open archived version from archive

  • Zenoss Community - Open Source Network Monitoring and Systems Management
    Thu 12 03 2015 10 36 0 by Mori Fri 12 18 2015 09 47 Normal topic Updating PythonCollector ZenPack 1 6 0 1 7 3 seems to break things by Jcurry Wed 12 09 2015 14 33 4 by Jcurry Tue 12 29 2015 05 36 Normal topic ZenPack Development Best Practice Documentation for Core 5 by Jenkinsk007 Fri 12 11 2015 11 45 3 by Jcurry Tue 12

    Original URL path: http://www.zenoss.org/forums/zenpack-development?order=title&sort=asc (2016-01-08)
    Open archived version from archive



  •