24 October, 2012

Cleanup DNS SRV records - Active Directory

As I mentioned in the previous post, when you perform DC metadata cleanup at 2 AM (after working all day with a hangover from the previous night) in a large IT environment, you don't want to walk through all SRV records and "folders" on the dnsmgmt.msc console and hit delete and then think about it 3 times, sweat a bit before clicking on Yes for the "Are you sure?" question.

What you need is a semi-automated way which creates the dnscmd commands for you to review on the fly and run - I wouldn't just run the dnscmd command blindly even if I'm about to fall asleep and my eyes are like arrowslits. The reason for this paranoia is that each SRV record has a so called RRData (Resource Record Data) which contains the DC name, weight, priority. If you specify only the SRV record for dnscmd and you don't specify the RRData, it wipes out the whole SRV record for ALL DCs not just for the one you wanted to clean up. Believe me, I've been there... or test it in a QA environment, you'll see.

Red circle shows the RRData, without it, dnsmcd will wipe out the record for ALL DCs!

Although restoring SRV records is fairly easy in a small environment: just restart the NETLOGON service on all DCs and they re-register all of them. However, if you have a global environment, with 100+ DCs in different parts of the world, it's a bit more difficult and can take several hours... and your weekend has gone and it's Monday again...

When you Think Big - think about large IT infrastructure with multiple 1000s of servers spread across 100+ locations - just take into account:
  • There's no such thing that "out of business hours" - when it's at night in EU for you, it's business hours in America. If it's Sunday for you, doesn't matter, people may work still in different part of the world 24x7.
  • You have to go through all 100+ DCs probably with a quick script but not just to restart the NETLOGON service, you will have to perform checkouts on all of the DCs to make sure all dependent services are running and are functional.
  • If you want to recover in a multi-site environment, it can take several hours to just replicate the SRV records you registered to all DCs/DNS servers.
  • Restarting the netlogon service may break some applications during business hours
  • etc.


Essentially, you want to make sure you delete what needs to be deleted and only that. Here is the main part of the script (without proper error handling, parameter handling, logging...etc.) which composes the dnscmd commands for deleting SRV records of a given domain controller:

$dcname = "c3poDC1"
$domain = "tatooine.com"
$dnsServer = "MyDNSServer"
$dnscommands = @()

# list all srv records containing the given DC name
$listSRVs = gwmi -Namespace root/microsoftdns -q "select * from MicrosoftDNS_SRvtype" -ComputerName $dnsServer | ?{($_.domainname -imatch $dcname) -or ($_.srvdomainname -imatch $dcname)}

#generate dnscmd syntax for each
$listSRVs | %{
   $tmpstr = $srvrecordname = $null

  
# if the textrepresentation contains our DC name,
   # then we can proceed
   if($_.textrepresentation -imatch $dcname){

     
# the srvrecord name is in the ownername property,
      # we need to delete the domain FQDN from it
      $srvrecordname = $_.ownername -replace $domain, ""
      # delete the domain FQDN from the SRV record name
      # and delete the text 'IN '
      $srvrecordNameandData = $_.textrepresentation -replace "((?<!.$domain.*).$domain)|IN ",""

      # store the command in a variable and in an array
      $tmpstr = "dnscmd $dnsServer /recorddelete $domain $srvrecordNameandData "
      $dnscommands += $tmpstr
   }
   else{
      write-host -ForegroundColor 'red' "Could not enumerate RRData from $srvrecordname"
   }
}
$dnscommands

Explanation:
  • The script enumerates all SRV records of a given DC
  • Picks up the 'textrepresentation' of each record
  • Creates the syntax of the dnscmd command for deleting that particular record

Clipboard friendly code:


 $dcname = "c3poDC1"  
 $domain = "tatooine.com"  
 $dnsServer = "MyDNSServer"  
 $dnscommands = @()  
   
 # list all srv records containing the given DC name  
 $listSRVs = gwmi -Namespace root/microsoftdns -q "select * from MicrosoftDNS_SRvtype" -ComputerName $dnsServer | ?{($_.domainname -imatch $dcname) -or ($_.srvdomainname -imatch $dcname)}   
 
 #generate dnscmd syntax for each  
 $listSRVs | %{  
      $tmpstr = $srvrecordname = $null  

      # if the textrepresentation contains our DC name, then we can proceed  
      if($_.textrepresentation -imatch $dcname){  

           # the srvrecord name is in the ownername property, we need to delete the domain FQDN from it  
           $srvrecordname = $_.ownername -replace $domain, ""  
  
           # delete the domain FQDN from the SRV record name and delete the text IN  
           $srvrecordNameandData = $_.textrepresentation -replace "((?<!.$domain.*).$domain)|IN ",""  

           # store the command in a variable and in an array  
           $tmpstr = "dnscmd $dnsServer /recorddelete $domain $srvrecordNameandData "  
           $dnscommands += $tmpstr  
      }  
      else{  
           write-host -ForegroundColor 'red' "Could not enumerate RRData from $srvrecordname"  
      }  
 }  
 $dnscommands  

May The Force...

19 October, 2012

List DC SRV records with PowerShell - Active Directory

To start off this blog, one of my friend suggested me posting a simple problem/solution and then continuing with deep dive stuff later. I'll take his advice.

Because this blog is supposed to show how real life problems make you think and how you can perform tasks in a large IT environment - which are easy to do on 10 servers but a bit more difficult on a large scale with 1000+ servers - let me start with the following use case which unfortunately happened with me too many times:

A domain controller dies, dies hard and you have to perform a metadata cleanup - this means you have to delete all references of the dead DC in the AD database and DNS. The last time this happened to me is because someone replaced the wrong disk in the only RAID 1 array of the server (one disk was broken and the other one wasn't until it was pulled out). The OS came back up but unfortunately the NTDS database was corrupt.
You can read more about metadata cleanup here. I'd like to focus on a specific step in this process, namely: DNS SRV record cleanup.

So when you do a metadata cleanup on a production AD at 2 AM, it's bad enough already. When you have 100+ domain controllers and you need to go through all DNS SRV records on dnsmgmt.msc to find out if the DC was truly removed and delete the lingering records that doesn't make it more fun.
I decided to decrease my stress level, so have a command that lists all srv records for a particular DC, here it is:

PS C:\> gwmi -Namespace root/microsoftdns -q "select * from MicrosoftDNS_SRvtype" -ComputerName MyDNSServer | ?{($_.domainname -imatch "c3poDC1") -or ($_.srvdomainname -imatch "c3poDC1")} | ft OwnerName,DomainName,srvdomainname -auto


OwnerNameDomainNamesrvdomainname
_kerberos._tcp.Mos_Eisley._sites.dc._msdcs.tatooine.comc3poDC1.tatooine.com.
_ldap._tcp.Mos_Eisley._sites.dc._msdcs.tatooine.comc3poDC1.tatooine.com.
_kerberos._tcp.dc._msdcs.tatooine.comc3poDC1.tatooine.com.
_ldap._tcp.dc._msdcs.tatooine.comc3poDC1.tatooine.com.
_kerberos._tcp.Mos_Eisley._sites.tatooine.comc3poDC1.tatooine.com.
_ldap._tcp.Mos_Eisley._sites.tatooine.comc3poDC1.tatooine.com.
_kerberos._tcp.tatooine.comc3poDC1.tatooine.com.
_kpasswd._tcp.tatooine.comc3poDC1.tatooine.com.
_ldap._tcp.tatooine.comc3poDC1.tatooine.com.
_kerberos._udp.tatooine.comc3poDC1.tatooine.com.
_kpasswd._udp.tatooine.comc3poDC1.tatooine.com.
_ldap._tcp.Mos_Eisley._sites.DomainDnsZones.tatooine.comc3poDC1.tatooine.com.
_ldap._tcp.DomainDnsZones.tatooine.comc3poDC1.tatooine.com.


Explanation:
  • MyDNSServer could be a domain controller if the DNS zone is AD integrated or it sits on domain controllers as a separate DNS zone
  • This oneliner filters for DomainName and for SRVDomainName properties as well ($_.domainname and $_.srvdomainname), this is because depending on what kind of DNS zone you have, different fields will contain the domain controller name in the WMI class:
    • If you have the DNS zone in a separate AD partition, the SRV records will be in a separate zone called _msdcs.domainfqdn, this will make the WMI class show the DC name in the SRVDomainName property
    • If you have a Windows 2000 AD compatible DNS zone, which means the zone is in the domain partition in AD, then the WMI class will have the DC name in the DomainName property

Prerequisites:
  • I assume you have Windows DNS in your environment
  • You are admin on your DNS server

Clipboard friendly code:
 PS C:\> gwmi -Namespace root/microsoftdns -q "select * from MicrosoftDNS_SRvtype" -ComputerName MyDNSServer | ?{($_.domainname -imatch "c3poDC1") -or ($_.srvdomainname -imatch "c3poDC1")} | ft OwnerName,DomainName,srvdomainname -auto  


In the next post, I'll share a way how to compose dnscmd command syntax out of this output to perform SRV record deletion quickly - instead of walking through all records on dnsmgmt.msc.

Until then, May the Force be you all.

t