LOGbinder Blog

Updates, Tips and News   RSS Feed  

How to analyze where events are coming from and how many

Thu, 31 Oct 2019 15:41:08 GMT

Recently we had an issue with a Supercharger customer/  They had 40+ distributed subscriptions and two of their many collectors were having latency issues. After hours and hours of investigation, research and troubleshooting we identified the issue. Of their tens of thousands of forwarders, just a few (yes not few thousand or few hundred but a handful) were generating a huge amount of events in comparison to the rest. This resulted in an unbalanced load balanced subscription. As a result of this troubleshooting, our Event Count By Source utility was born. Just download the utility and run it on a collector against some of the logs. You will see how it totals events by source and sorts them in descending order.

This utility is intended to help you determine where forwarded events are coming from and in what quantity. It analyzes the specified log and counts events by source computer and source log. 3 tab delimited files are produced in the current directory named after the type of data, log name, and computer name allowing you to co-mingle output from multiple executions of the program.

The 3 files summarize event counts by source machine only, source log only and by log and machine. A 4th file simply documents the number of events in the log.

There is only one parameter - log name. For a list of logs run "get-winevent -listlog *".
Omit log name and it will default to ForwardedEvents.

Download Event Count By Source utility.

This is a free utility from LOGbinder (www.logbinder.com) which is a division of Monterey Technology Group, Inc.
(c) 2019 Monterey Technology Group, Inc (MTG). You are free to use and copy this program for lawful uses. Use at your own risk. By your use of this program you hold MTG harmless of any untoward results.

Kerbpurge - Group memebership without a reboot

Sat, 01 Jun 2019 11:30:18 GMT

KerbPurge - What is it?

When you add a computer to a group in Active Directory, the computer does not know that it has been added to the group until a reboot happens. There are many obvious reasons why this is a problem. For example, you can't just reboot production servers at any time. Most organizations have some sort process in place for scheduling server reboots which in itself can be a time consuming process. When it comes to Windows Event Collection there are many reasons for adding endpoint forwarders to groups, especially if you are using Supercharger. For example, Superchargers builtin load balanced or distributed subscription feature rely's on group changes to keep forwarders balanced across the number of specified collectors. This is why Randy Franklin Smith of UltimateITSecurity.com designed and wrote KerbPurge.

Benefits and Features

  • Safely and efficiently make Windows computers see group membership changes
  • Tiny Windows service
  • Installable via Group Policy's Software Installation feature
  • Only purges tickets for the Network Service logon session and only when group membership has been changed for a computer
  • No measurable resource usage

Installation and Configuration

To install and configure Kerbpurge visit our knowledgebase for the step-by-step guide.

    Using Site-linked GPOs for Targeting Windows Event Collectors Prevents Forwarder Health Analysis

    Wed, 22 Nov 2017 10:12:47 GMT

    For the most part WEC allows you to control event forwarding from the collector but there is one setting in group policy: Group Policy Management Editor\Default Domain Policy\Computer Configuration\Policies\Administrative Templates\Windows Components\Event Forwarding\"Configure target Subscription Manager setting Enabled

    This setting is like the bootstrapper for Windows Event Collection.  When computers out in the domain apply group policy and see this setting, they now know which collector(s) to start checking in with regularly to find new/updated subscriptions.

    In larger environments with regional data centers we often setup local collectors so that we keep event forwarding traffic local – computer in a given region send events to the local collector.  That makes total sense but the possible problem arises with how you do this with group policy.

    You may have Sites already defined in Active Directory that correspond to each region of computers and their local collector.  Active Directory allows you to assign GPOs to Sites and then each computer physically in that site will apply the policies in that GPO – a seemingly convenient way to configure computers based on their location. 

    The problem arises when you try to assess the health of WEC subscriptions.  Before you can determine if all computers are actively forwarding events for a given subscription you need to know which computer should be sending events.  Supercharger’s Deterministic forwarder health analysis automates this process for you.  By default, when you specify Deterministic mode on a subscription, Supercharger looks at the groups assigned to that subscription, enumerate the computer accounts there-in and uses that as the baseline for the computers that should be sending events.  Then Supercharger compares that list to the list of computers WEC says are currently sending events.  (This is a simplified description, click here for more information). 

    But sometimes we encounter environments where the customer has specified an all-encompassing group like Domain Computers which includes, well, all computers in the domain – probably many more times the number of computers in the region of that collector.  WEC doesn’t care: only those computers in that group which also apply the GPO pointing to a given collector will subscribe and start sending events.  But we obviously can’t use the group’s membership to perform health analysis.

    For this reason, we added an alternative method to Deterministic health analysis that allows you to specify a custom LDAP filter.  This allows you to use most any attribute available on computer accounts to determine the baseline list of computers that should be sending events.  For instance, if your computer naming convention prefixes computers with region you could use that.  Or maybe department or description on the computer account provides the appropriate criteria – or a combination of fields.  The bottom line is you specify an ldap filter that produces the right set of computers for that collector and assign it to each subscription on the collector.

    Here’s the gotcha – look at the attributes available on computer accounts in AD – there’s nothing for Site because it's not static.  Whenever a computer boots up it compares it's IP address to Site definitions in AD to determine which Site it's in.  So, you can't specify an LDAP filter where "site=Detroit" for example. 

    If you want to use deterministic health analysis for accurate diagnosis of which computers are sending events and which ones where forwarding is broken for some reason you need an OU or some other criteria in AD that can be specified in an LDAP filter. 

    If there’s no such data on your computer accounts you might consider creating a Startup script in the same GPO that causes each computer to update a selected attribute on its computer account in AD with its current site.  You’d need to make sure to delegate Write access that attribute on computer accounts.  The script would need to query Windows for its current site and then update the appropriate field on its computer account. 

    Failing that, if you have a good idea of the number of computers that should be sending events you can use Arbitrary instead of Deterministic.  Or simply use Empirical instead of Deterministic.


    In-depth How To's for Windows Event Collection

    Mon, 28 Aug 2017 13:36:35 GMT

    Here at LOGbinder we have been deep in the weeds with Windows Event Forwarding/Collection (WEC or WEF) for quite some time now.  In the past month since we’ve released Supercharger for WEC and opened our new forums for WEC and also Supercharger.  This has resulted in many of you asking questions and finding new challenges with WEC. 

    For example, a few Supercharger users have recently asked about workstation status outside of work hours like weekends and holidays.  As you know, one of the benefits of a Supercharged WEC environment is being able to see the health status of all your forwarders.  During weekends or holidays, a forwarder may be shutdown or sleeping until the user gets the machine online again.  During this workstation’s down time you don’t want a healthy workstation reporting as unhealthy in Supercharger.  This KB article explains a how to change a simple setting to keep this from happening.

    Another situation that some users are dealing with is when they need to define expected forwarders by some AD criteria other than an AD group.  For example, you create a subscription targeting “Domain Computers” but you only want a subset of the computers running Windows 10 in this group to forward events.  We have had users scratching their heads trying to figure out if this is possible without creating new AD groups which can take time, especially if you are working with thousands of forwarders like some of our users.  This KB article explains how to do this using LDAP filters in Superchargers Deterministic Subscription Policies.

    We will do our best to keep you updated with tips and tricks to get your WEC Supercharged.  In the meantime, feel free to browse the “How To” section in our Support Portal to see if you are missing out on any of our latest articles and tips.

    Randy releases two new "How-To" Videos

    Wed, 21 Jun 2017 13:55:45 GMT
    Randy Franklin Smith, guru at UltimateWindowsSecurity.com, just released two new "How-To" video's on monitoring two important areas with Windows Event Collection.

    Video 1 - In this 4 minute video, Randy shows you step-by-step how you can use Supercharger to create a WEC susbscription that pulls PowerShell security events from all of your endpoints to a central collector.

    Video 2 - In this 8 minute video, Randy shows you how to monitor security event ID 4688 from all of your endpoints. Obviously this would normally create a plethora of data but using Supercharger's Common System Process noise filter you will see how you can leave 60% of the noise at the source.

    You can watch the video's by clicking on the links above or visiting the resources page for Supercharger by clicking here.

    previous | next

    powered by Bloget™