2011-10-05
2011-09-28
Dynamic views from Blogger
Google's blogger team today announced a pretty cool feature that they call "Dynamic views". Announcement link is here. I am trying 'magazine' view right now. So far so good, none of the special formatting and template customizations I had done are broken. Way to go Blogger Team!
2011-09-14
What's coming in Server 8
In his latest newsletter, Mark Minasi has a wonderful summary of features coming in Windows Server 8 from BUILD event that took place this week.
Windows 8 Dynamic Access Control (DAC) seems quite interesting and is a clear indication that Microsoft is trying to respond to an every day problem of 'permissions and auditing' in large enterprises. To be honest, I am not sure tagging is the answer to it, mainly because it's an attempt to use some of the unused attributes in AD and in file tags in NTFS, which may prove to be limited once enterprises starts being creative in employing the technology. However, the simple fact that it will be possible to use Regular Expressions on file ACLs is a welcome news.
We will have to wait and see the implementation details. In the meantime here is an article published today at Windows IT Pro by Sean Deuby that explains the DAC in more details.
PowerShell is of course getting a bigger pie in server management (e.g. Active Directory Admin Center a.k.a. ADAC) with Version 3. Number of cmdlets are going from ~300 to 2300!
Speaking of AD, there does not seem to be much news other than making it Virtualization friendly. If you are still waiting for a SQL/Database driven directory, don't! It's not coming yet!
Windows 8 Dynamic Access Control (DAC) seems quite interesting and is a clear indication that Microsoft is trying to respond to an every day problem of 'permissions and auditing' in large enterprises. To be honest, I am not sure tagging is the answer to it, mainly because it's an attempt to use some of the unused attributes in AD and in file tags in NTFS, which may prove to be limited once enterprises starts being creative in employing the technology. However, the simple fact that it will be possible to use Regular Expressions on file ACLs is a welcome news.
We will have to wait and see the implementation details. In the meantime here is an article published today at Windows IT Pro by Sean Deuby that explains the DAC in more details.
PowerShell is of course getting a bigger pie in server management (e.g. Active Directory Admin Center a.k.a. ADAC) with Version 3. Number of cmdlets are going from ~300 to 2300!
Speaking of AD, there does not seem to be much news other than making it Virtualization friendly. If you are still waiting for a SQL/Database driven directory, don't! It's not coming yet!
2011-09-12
Writing Binary Data to Registry
Uh, oh! I found yet another post in drafts from 2007. I do not recall the events but posting it for common good :)
**************************************************************
Yesterday, a friend from work showed me an interesting script he was working on. His script was reading a reg_binary type registry key, modifying its value and was 'attempting' to write it back to registry.
There was an issue with 'writing back to registry'. He was using SetBinaryValue method to write an array, which had modified values, back to registry but vbscript kept on complaining there was a "type mismatch" for this line:
if he set the arrValues to a static array like this
script worked without any issues.
I took the code and tried to figure out what was wrong with it. I would like to write down a couple of key points for those people who are trying to do something similar.
* When we are talking about Binary data in Registry, we are actually referring to Hexadecimal values, because that's the Registry-speak (1984 anyone?). We can use GetBinaryValue Method of WMI's StdRegProv class. Output is "an array of binary bytes"
**************************************************************
Yesterday, a friend from work showed me an interesting script he was working on. His script was reading a reg_binary type registry key, modifying its value and was 'attempting' to write it back to registry.
There was an issue with 'writing back to registry'. He was using SetBinaryValue method to write an array, which had modified values, back to registry but vbscript kept on complaining there was a "type mismatch" for this line:
Return = oReg.SetBinaryValue(HKEY_LOCAL_MACHINE, strKeyPath & "\" & subKey, strValueName, arrValues
if he set the arrValues to a static array like this
arrValues=Array(1,2,3,4,5,6)
script worked without any issues.
I took the code and tried to figure out what was wrong with it. I would like to write down a couple of key points for those people who are trying to do something similar.
* When we are talking about Binary data in Registry, we are actually referring to Hexadecimal values, because that's the Registry-speak (1984 anyone?). We can use GetBinaryValue Method of WMI's StdRegProv class. Output is "an array of binary bytes"
Also, although registry speaks in hex as far as binary data is concerned, "SetBinaryValue" method does not understand Hex
Consider the following Reg Key/Value (pasting from exported .reg):
Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Adobe Acrobat\7.0\FeatureLockDown\cDefaultLaunchURLPerms]" sSchemePerms2"=hex:76,65,72,73,69,6f,6e,3a,31,7c,73,68,65,6c,6c,3a,33,7c,68,\ 63,70,3a,33,7c,6d,73,2d,68,65,6c,70,3a,33,7c,6d,73,2d,69,74,73,3a,33,7c,6d,\ 73,2d,69,74,73,73,3a,33,7c,69,74,73,3a,33,7c,6d,6b,3a,33,7c,6d,68,74,6d,6c,\ 3a,33,7c,68,65,6c,70,3a,33,7c,64,69,73,6b,3a,33,7c,61,66,70,3a,33,7c,64,69,\ 73,6b,73,3a,33,7c,74,65,6c,6e,65,74,3a,33,7c,73,73,68,3a,33,7c,6a,61,76,61,\ 73,63,72,69,70,74,3a,31,7c,76,62,73,63,72,69,70,74,3a,31,7c,61,63,72,6f,62,\ 61,74,3a,32,7c,6d,61,69,6c,74,6f,3a,32,7c,66,69,6c,65,3a,32,00
If we convert it to string, we get something like
version:1shell:3hcp:3ms-help:3ms-its:3 ms-itss:3its:3mk:3mhtml:3help:3disk:3 afp:3disks:3telnet:3ssh:3javascript:1 vbscript:1acrobat:2mailto:2 file:2
Then, you will need to change
mailto:2to
mailto:3with vbscript's REPLACE funtion
sNewValue = replace(sOldValue,"mailto:2","mailto:3", 1, -1 , 1)
* SetBinaryValue method is used to write "an array of binary data values" to registry. What is misleading here, and this was the key to solving our issue, is that method actually needs a variant or you will get type mismatch.
So this code works :
'Assumes objRegistry is a valid StdRegProv object.On Error Resume Next Const HKEY_LOCAL_MACHINE As Long = &H80000002 Dim lRC As Long Dim sPath As String Dim uBinary() As Variant sPath = "SOFTWARE\MyKey" uBinary = Array(1,2,3,4,5,6,7,8) lRC = objRegistry.SetBinaryValue(HKEY_LOCAL_MACHINE, sPath, "MyBinaryNamedValue", uBinaryData) If (lRC = 0) And (Err.Number = 0) Then 'Do something Else 'An error occurred End If* Pay attention to Array function, which returns a Variant containing an array as mentioned in MS documentation:
"A variable that is not declared as an array can still contain an array. Although a Variant variable containing an array is conceptually different from an array variable containing Variant elements, the array elements are accessed in the same way."
Ubuntu Update Manager fails to download packages
I am having an issue with my Ubuntu installation (11.04) where once laptop is suspended, it never wakes-up. I can repro this by simply putting it to sleep by hitting Fn+F4. The only solution I could find is pressing Power button for 5 secs to completely power it off.
I checked Bug reports at launchpad.net. Although I see several people reported it, they see it on different hardware. So, I will file a bug report but wanted to make sure I have all the updates.
When I brought up "Update Manager", it showed me a couple of updates but when I click to install them I got an error:
"failed to download packages, check you internet connection".
Message may be a bit misleading as it suggest connectivity is the issue but in fact solution was simply clicking "Check" button to refresh list of available updates.
I checked Bug reports at launchpad.net. Although I see several people reported it, they see it on different hardware. So, I will file a bug report but wanted to make sure I have all the updates.
When I brought up "Update Manager", it showed me a couple of updates but when I click to install them I got an error:
"failed to download packages, check you internet connection".
Message may be a bit misleading as it suggest connectivity is the issue but in fact solution was simply clicking "Check" button to refresh list of available updates.
2011-09-08
Fraudulent Certificates...Again
If you did not hear about the latest saga of "rogue certificates" out there because of a Dutch company called DigiNotar by now, well, you are not paying attention to the security news :)
Certificate Authorities are backbone of Trust system we use for "secure" online access. To see that "lock" icon in the browser when we visit a site with an SSL certificate, and even a green bar if the site has an Extended Validation SSL (EV SSL), may give us a sense of security, which unfortunately proving to be a "false sense of security" these days.
There are plenty of articles out there on what happened (even a Wiki), how it happened, who got involved and what Microsoft, Google, Mozilla, etc are doing to contain damage, even a Wiki about it but also what you should be aware of. Here is one from Windows Secrets that explains it in laymen terms. I personally liked the detailed account from Firefox folks.
If you are reading this blog, you are probably interested in an easier way to find whether you have a cert or not and PowerShell can come to the rescue:
I do not have it on my machine, so I won't go further but you if you search only "Digi", you will see some results:
Unfortunately, Safari / OS X does not have a mechanism to detect Revoke Lists (RL) but Apple should be releasing an update soon to fix the chain. In the mean time you can open up 'Keychain Access' tool and remove the DigiNotarRoot Certificate from GUI but where is the fun in that?
If you double click the certificate, you get detailed information as shown below.
And what if you had to do this on multiple Macs in an enterprise environment? You would want to use command line to do it. The command for all certificate related work is named 'security'.
You can dump a pretty list of all Root CAs in OS X using 'dump-keychain' parameter of 'security' command, which, as we mentioned above, is used to manipulate Keychains from command line.
If we only wanted to display the Friendly names of certificates, which would be equivalent of what we see in 'Keychain Access' GUI, we can filter by 'labl'
We can filter the results that start with 'D' and while at it, beautify it by getting rid of '=' and everything before it:
We can use also use find-certificate parameter to find the certificate and print all info. (-a for all keychains, not actually necessary here as we know this is a Root Certificate but good to be safe. If you happen to know the e-mail you could also use -e parameter)
Well, enough playing. To delete the certificate, we will use 'delete-certificate' command. We have two choices:
1) Use -c parameter which is using 'common name'
2) Use SHA-1 fingerprint (safer).
Let's do both.
1) We need to use common name. This is the name you see in the GUI and we seem to get it from 'labl' line above. Command becomes
2) As mentioned above, using SHA-1 fingerprint is less error-prone than relying on common names. To do that, we have to first locate the fingerprint. Noticed that it was not showing above when we displayed the certificate?
OK, so how do we get the fingerprint? Simple: we add -Z to the 'find-certificate' command which returns SHA-1 in the first line and then prints what we have seen above. So we will simply 'grep' the fingerprint:
And now we can get rid of the certificate:
Well that's all. Now, all you would need is to put these two lines in a shell script (find fingerprint if certificate exists & delete it), then run it against all your Macs.
Note that in general Safari and Chrome honor system-wide certificates, however, some versions of Firefox is not using the Keychain to store/retrieve certificates. It has its own database and you might need to manipulate that as well. Also note that browsers have their own list of trusted CAs so updating Chrome and Firefox would solve the problem as of today.
Update: There are several articles about why revoking this certificate may not be enough because of the way EV SSL treated in Safari. If you delete the certificate, this should not be a concern. However, there seems to be additional certificates that needs to be revoked to be safer.
In any case, I visited DigiNotar web site on Safari and found a page with "Order" button that takes you to another page with SSL. As soon as I clicked the "Order" link, Safari warned me that the certificate on the site was invalid (expired).
As far as I can tell chain goes like this:
DigiNotar Root CA -> DigiNotar Services 1024 CA -> *.diginotar.nl
So this seems to be good sign. I tried some other sites but I am yet to find a site that was issued and SSL from the the Root CA I deleted.
Update2: I saw https://www.maestre.com mentioned as a test site here. This is what I got on Safari:
It's good that I am getting a notification. Unfortunately, it is not because DigiNotar Root CA is missing from my Root keychain but because certificate has expired.
Update3: Apple today released a security patch to put the issue at rest. In their terms, here is what they did:
Description: Fraudulent certificates were issued by multiple certificate authorities operated by DigiNotar. This issue is addressed by removing DigiNotar from the list of trusted root certificates, from the list of Extended Validation (EV) certificate authorities, and by configuring default system trust settings so that DigiNotar's certificates, including those issued by other authorities, are not trusted.
Of course, they would not tell exactly what they did. I had a pretty good idea which file they were talking about. Let's look at the last one "configuring default system trust settings":
So backed up these files before applying the patch and ran a diff. First let's look at a record in the last one.
The bottom is the original, and above you see the updated file. Basically they updated the date and added a new array with a dictionary which set kSecTrustSettingsResult to 3.
Notice that I am looking at the Key that starts with C060E... which is the SHA-1 fingerprint we got above.
Also notice that IssuerName, we know that's DigiNotar. As noted in some of the links above, there were several certs by DigiNotar published by others. I can tell from the diff which are the ones that were affected but I cannot tell a way of figuring them out only by looking at the original as Issuer Names would be different and don't have a list of chains where DigiNotar exists.
Certificate Authorities are backbone of Trust system we use for "secure" online access. To see that "lock" icon in the browser when we visit a site with an SSL certificate, and even a green bar if the site has an Extended Validation SSL (EV SSL), may give us a sense of security, which unfortunately proving to be a "false sense of security" these days.
There are plenty of articles out there on what happened (even a Wiki), how it happened, who got involved and what Microsoft, Google, Mozilla, etc are doing to contain damage, even a Wiki about it but also what you should be aware of. Here is one from Windows Secrets that explains it in laymen terms. I personally liked the detailed account from Firefox folks.
If you are reading this blog, you are probably interested in an easier way to find whether you have a cert or not and PowerShell can come to the rescue:
PS C:\Users\Adil> gci certificate::LocalMachine\Root |?{$_.subject -match "DigiNotar"}
I do not have it on my machine, so I won't go further but you if you search only "Digi", you will see some results:
PS C:\Users\Adil> gci certificate::LocalMachine\Root |?{$_.subject -match "Digi"}
Directory: Microsoft.PowerShell.Security\Certificate::LocalMachine\Root
Thumbprint Subject
---------- -------
5557C0953FBD9F93745B214FB2483E9369B597F0 CN=DT Soft Ltd, OU=Digital ID Class 3 - Microsoft S
5FB7EE0633E259DBAD0C4C9AE6D38F1A61C7DC25 CN=DigiCert High Assurance EV Root CA, OU=www.digic
3E2BF7F2031B96F38CE6C4D8A85D3E2D58476A0F CN=StartCom Certification Authority, OU=Secure Digi
0563B8630D62D75ABBC8AB1E4BDFB5A899B24D43 CN=DigiCert Assured ID Root CA, OU=www.digicert.com
Unfortunately, Safari / OS X does not have a mechanism to detect Revoke Lists (RL) but Apple should be releasing an update soon to fix the chain. In the mean time you can open up 'Keychain Access' tool and remove the DigiNotarRoot Certificate from GUI but where is the fun in that?
If you double click the certificate, you get detailed information as shown below.
And what if you had to do this on multiple Macs in an enterprise environment? You would want to use command line to do it. The command for all certificate related work is named 'security'.
You can dump a pretty list of all Root CAs in OS X using 'dump-keychain' parameter of 'security' command, which, as we mentioned above, is used to manipulate Keychains from command line.
If we only wanted to display the Friendly names of certificates, which would be equivalent of what we see in 'Keychain Access' GUI, we can filter by 'labl'
adil$ security dump-keychain "/System/Library/Keychains/SystemRootCertificates.keychain" |grep labl "labl"<blob>="Prefectural Association For JPKI" "labl"<blob>="Entrust.net Certification Authority (2048)" "labl"<blob>="A-Trust-Qual-01" "labl"<blob>="A-Trust-Qual-02" "labl"<blob>="A-Trust-nQual-01" "labl"<blob>="A-Trust-nQual-03" "labl"<blob>="AOL Time Warner Root Certification Authority 1" "labl"<blob>="AOL Time Warner Root Certification Authority 2" ...
We can filter the results that start with 'D' and while at it, beautify it by getting rid of '=' and everything before it:
adil$ security dump-keychain "/System/Library/Keychains/SystemRootCertificates.keychain" |grep labl |awk -F '=' '{print $2}' |grep ^\"D
"DST Root CA X4"
"Deutsche Telekom Root CA 2"
"DigiCert Assured ID Root CA"
"DigiCert Global Root CA"
"DigiCert High Assurance EV Root CA"
"DigiNotar Root CA"
"DoD CLASS 3 Root CA"
"DoD Root CA 2"
"DST Root CA X3"
"DST ACES CA X6"
We can use also use find-certificate parameter to find the certificate and print all info. (-a for all keychains, not actually necessary here as we know this is a Root Certificate but good to be safe. If you happen to know the e-mail you could also use -e parameter)
adil$ security find-certificate -a -c "DigiNotar"/System/Library/Keychains/SystemRootCertificates.keychain keychain: "/System/Library/Keychains/SystemRootCertificates.keychain" class: 0x80001000 attributes: "alis"<blob>="info@diginotar.nl" "cenc"<uint32>=0x00000003 "ctyp"<uint32>=0x00000001 "hpky"<blob>=0x8868BFE08E35C43B386B62F7283B8481C80CD74D "\210h\277\340\2165\304;8kb\367(;\204\201\310\014\327M" "issu"<blob>=0x305F310B3009060355040613024E4C31123010060355040A1309444947494E4F544152311A301806035504031311444947494E4F54415220524F4F542043413120301E06092A864886F70D0109011611696E666F40646967696E6F7461722E6E6C "0_1\0130\011\006\003U\004\006\023\002NL1\0220\020\006\003U\004\012\023\011DIGINOTAR1\0320\030\006\003U\004\003\023\021DIGINOTAR ROOT CA1 0\036\006\011*\206H\206\367\015\001\011\001\026\021info@diginotar.nl" "labl"<blob>="DigiNotar Root CA" "skid"<blob>=0x8868BFE08E35C43B386B62F7283B8481C80CD74D "\210h\277\340\2165\304;8kb\367(;\204\201\310\014\327M" "snbr"<blob>=0x0C76DA9C910C4E2C9EFE15D058933C4C "\014v\332\234\221\014N,\236\376\025\320X\223<L" "subj"<blob>=0x305F310B3009060355040613024E4C31123010060355040A1309444947494E4F544152311A301806035504031311444947494E4F54415220524F4F542043413120301E06092A864886F70D0109011611696E666F40646967696E6F7461722E6E6C "0_1\0130\011\006\003U\004\006\023\002NL1\0220\020\006\003U\004\012\023\011DIGINOTAR1\0320\030\006\003U\004\003\023\021DIGINOTAR ROOT CA1 0\036\006\011*\206H\206\367\015\001\011\001\026\021info@diginotar.nl"
Well, enough playing. To delete the certificate, we will use 'delete-certificate' command. We have two choices:
1) Use -c parameter which is using 'common name'
2) Use SHA-1 fingerprint (safer).
Let's do both.
1) We need to use common name. This is the name you see in the GUI and we seem to get it from 'labl' line above. Command becomes
adil$ sudo security delete-certificate -c "DigiNotar Root CA" /System/Library/Keychains/SystemRootCertificates.keychain
2) As mentioned above, using SHA-1 fingerprint is less error-prone than relying on common names. To do that, we have to first locate the fingerprint. Noticed that it was not showing above when we displayed the certificate?
OK, so how do we get the fingerprint? Simple: we add -Z to the 'find-certificate' command which returns SHA-1 in the first line and then prints what we have seen above. So we will simply 'grep' the fingerprint:
adil$ security find-certificate -a -c "DigiNotar" -Z /System/Library/Keychains/SystemRootCertificates.keychain |grep SHA-1 SHA-1 hash: C060ED44CBD881BD0EF86C0BA287DDCF8167478C
And now we can get rid of the certificate:
adil$ sudo security delete-certificate -Z C060ED44CBD881BD0EF86C0BA287DDCF8167478C /System/Library/Keychains/SystemRootCertificates.keychain
Well that's all. Now, all you would need is to put these two lines in a shell script (find fingerprint if certificate exists & delete it), then run it against all your Macs.
Note that in general Safari and Chrome honor system-wide certificates, however, some versions of Firefox is not using the Keychain to store/retrieve certificates. It has its own database and you might need to manipulate that as well. Also note that browsers have their own list of trusted CAs so updating Chrome and Firefox would solve the problem as of today.
Update: There are several articles about why revoking this certificate may not be enough because of the way EV SSL treated in Safari. If you delete the certificate, this should not be a concern. However, there seems to be additional certificates that needs to be revoked to be safer.
In any case, I visited DigiNotar web site on Safari and found a page with "Order" button that takes you to another page with SSL. As soon as I clicked the "Order" link, Safari warned me that the certificate on the site was invalid (expired).
As far as I can tell chain goes like this:
DigiNotar Root CA -> DigiNotar Services 1024 CA -> *.diginotar.nl
So this seems to be good sign. I tried some other sites but I am yet to find a site that was issued and SSL from the the Root CA I deleted.
Update2: I saw https://www.maestre.com mentioned as a test site here. This is what I got on Safari:
It's good that I am getting a notification. Unfortunately, it is not because DigiNotar Root CA is missing from my Root keychain but because certificate has expired.
Update3: Apple today released a security patch to put the issue at rest. In their terms, here is what they did:
Description: Fraudulent certificates were issued by multiple certificate authorities operated by DigiNotar. This issue is addressed by removing DigiNotar from the list of trusted root certificates, from the list of Extended Validation (EV) certificate authorities, and by configuring default system trust settings so that DigiNotar's certificates, including those issued by other authorities, are not trusted.
Of course, they would not tell exactly what they did. I had a pretty good idea which file they were talking about. Let's look at the last one "configuring default system trust settings":
adil$ pwd
/System/Library/Keychains
adil$ ls -l
total 1048
-rw-r--r-- 1 root wheel 5353 Sep 9 17:53 EVRoots.plist
-rw-r--r-- 1 root wheel 167848 Jul 9 23:39 SystemCACertificates.keychain
-rw-r--r-- 1 root wheel 395312 Sep 9 17:53 SystemRootCertificates.keychain
-rw-r--r-- 1 root wheel 86380 Sep 9 17:53 SystemTrustSettings.plist
-rw-r--r-- 1 root wheel 282984 Jul 28 2008 X509Anchors
The bottom is the original, and above you see the updated file. Basically they updated the date and added a new array with a dictionary which set kSecTrustSettingsResult to 3.
Notice that I am looking at the Key that starts with C060E... which is the SHA-1 fingerprint we got above.
Also notice that IssuerName, we know that's DigiNotar. As noted in some of the links above, there were several certs by DigiNotar published by others. I can tell from the diff which are the ones that were affected but I cannot tell a way of figuring them out only by looking at the original as Issuer Names would be different and don't have a list of chains where DigiNotar exists.
2011-09-02
Find your Video Driver version with PowerShell
Well, I wrote this quite sometime ago (Bonus for geeks! Can you tell from the driver version?) but apparently forgot to post it:
Here is one way you can use WMI and PowerShell to get the version of driver you have installed for your video card(s).
gwmi is shorthand for get-wmi.
I happen to have two ATI cards. It's not really necessary to select Device ID and name. You can simplify it as follows:
Why parentheses? Because that way you can access the properties of an object.
Why [0]? Well, b/c it's an array and you know you have the same card. So, it's enough to get driver version of the first card.
How did I know that I have to use Win32_VideoController WMI class? Well, I did not but there is no black magic here, just a bit of guess work and good ol' trial & error:
Here is one way you can use WMI and PowerShell to get the version of driver you have installed for your video card(s).
PS C:\> gwmi win32_VideoController |select DeviceID,Name,DriverVersion |ft -a DeviceID Name DriverVersion -------- ---- ------------- VideoController1 ATI Radeon HD 5700 Series 8.812.0.0 VideoController2 ATI Radeon HD 5700 Series 8.812.0.0
gwmi is shorthand for get-wmi.
I happen to have two ATI cards. It's not really necessary to select Device ID and name. You can simplify it as follows:
PS C:\> (gwmi win32_VideoController)[0].DriverVersion 8.812.0.0
Why parentheses? Because that way you can access the properties of an object.
Why [0]? Well, b/c it's an array and you know you have the same card. So, it's enough to get driver version of the first card.
How did I know that I have to use Win32_VideoController WMI class? Well, I did not but there is no black magic here, just a bit of guess work and good ol' trial & error:
PS C:\> gwmi -list |?{$_ -match "video"}
NameSpace: ROOT\cimv2
Name Methods Properties
---- ------- ----------
CIM_VideoBIOSElement {} {BuildNumber, Caption, CodeSet, Description...}
CIM_VideoController {SetPowerState, R... {AcceleratorCapabilities, Availability, CapabilityDescripti...
CIM_PCVideoController {SetPowerState, R... {AcceleratorCapabilities, Availability, CapabilityDescripti...
Win32_VideoController {SetPowerState, R... {AcceleratorCapabilities, AdapterCompatibility, AdapterDACT...
CIM_VideoBIOSFeature {} {Caption, CharacteristicDescriptions, Characteristics, Desc...
CIM_VideoBIOSFeatureVideoBIOSEle... {} {GroupComponent, PartComponent}
CIM_VideoSetting {} {Element, Setting}
Win32_VideoSettings {} {Element, Setting}
CIM_VideoControllerResolution {} {Caption, Description, HorizontalResolution, MaxRefreshRate...
Win32_VideoConfiguration {} {ActualColorResolution, AdapterChipType, AdapterCompatibili...
Guess which Python string find method is faster?
I came across a question on finding which of the two simple string find methods was faster. So, let's play a game. All we are trying to determine is whether a single character ('ch') passed to our function is lowercase or not. Can you guess which method will be fastest out of these four?
Clearly, you can guess the first one will be the sore loser. It is using a string function (string.find) on all the possible lowercase characters (string.lowercase) to check if the passed character matches one. 'Find' Function will return -1, if it cannot find the passed character, that's why result is compared against '-1'. OK, but how about the rest?
is_lower2 function is also using a string function (lower) to lower only the passed character and is then comparing it against its original value. So, basically there are two operations here, but no iteration as in find.
is_lower3 is using 'in' operator against all possible lower case values. So, our string operation here is to list all possible values with (string.lowercase). Is this faster than is_lower2?
is_lower4 is comparing the passed character against the boundaries of lower case letters. There are no iterations or string operations as before but two comparison operations. That should be fast, right? Note that we are using Ascii characters here for comparison. If you print string.lowercase, 'z' is not the last character, it's '\xff' which looks like 'y' with two dots over it on my PC, but be assured that results are not affected any noticeable way.
So, let's timeit :
You probably guessed it but here are results to prove our hunch on which string search method is faster:
Did you guess it right?
# check result of string find function def is_lower1(ch): return (string.find(string.lowercase, ch) !=-1)
## compare the string char to lower case version of it def is_lower2(ch): return (ch.lower() == ch)
# check string char against all lowercase chars def is_lower3(ch): return (ch in string.lowercase)
# check the char against the lowercase boundries def is_lower4(ch): return 'a' <= ch <= 'z'
Clearly, you can guess the first one will be the sore loser. It is using a string function (string.find) on all the possible lowercase characters (string.lowercase) to check if the passed character matches one. 'Find' Function will return -1, if it cannot find the passed character, that's why result is compared against '-1'. OK, but how about the rest?
is_lower2 function is also using a string function (lower) to lower only the passed character and is then comparing it against its original value. So, basically there are two operations here, but no iteration as in find.
is_lower3 is using 'in' operator against all possible lower case values. So, our string operation here is to list all possible values with (string.lowercase). Is this faster than is_lower2?
is_lower4 is comparing the passed character against the boundaries of lower case letters. There are no iterations or string operations as before but two comparison operations. That should be fast, right? Note that we are using Ascii characters here for comparison. If you print string.lowercase, 'z' is not the last character, it's '\xff' which looks like 'y' with two dots over it on my PC, but be assured that results are not affected any noticeable way.
So, let's timeit :
if __name__ == '__main__':
import string
from timeit import Timer
t = Timer("is_lower1('A')", "from __main__ import is_lower1")
print "is_lower1 result: %f" % t.timeit()
t = Timer("is_lower2('A')", "from __main__ import is_lower2")
print "is_lower2 result: %f" % t.timeit()
t = Timer("is_lower3('A')", "from __main__ import is_lower3")
print "is_lower3 result: %f" % t.timeit()
t = Timer("is_lower4('A')", "from __main__ import is_lower4")
print "is_lower4 result: %f" % t.timeit()
You probably guessed it but here are results to prove our hunch on which string search method is faster:
is_lower1 result: 0.957694 is_lower2 result: 0.322355 is_lower3 result: 0.256491 is_lower4 result: 0.201267
Did you guess it right?
2011-08-26
Fedora 15
It's been five years since I switched from Fedora to Ubuntu. I wanted to see where Fedora is these days, so downloaded and run Fedora 15 (F15) Live CD. A couple of first impression notes below...
Installation:
I let the live CD boot and run F15. It comes with Gnome 3.0 and here is a link to Gnome 3.0 cheat sheet. I then chose to install it on disk using the link in "Activities" > "Applications".
I chose to partition manually, allowed 500MB for /Boot and 30GB for / as I wanted to use the rest for Ubuntu.
Installation was quick. Wizard is designed to warn on things like missing /swap partition, weak passwords etc. I also liked the fact that it was able to detect Time Zone correctly (in contrast to Macs usually defaulting to West Coast and requiring me to choose East Coast). Good job overall.
Software Update:
ISOs are not frequently updated but when I install Ubuntu it checks with its repos as soon as internet connectivity is established and almost immediately Software Update icon is displayed. I waited a bit expecting the same thing to happen with Fedora, it did not. So I ran "Software Update" and of course there were tons of updates available. Lo and behold, I got a cryptic "Transaction error" message as soon as I clicked update:
"Transaction error could not add package update for fedora-release-rawhide-15-3(noarch)updates: fedora-release-rawhide-15-3.noarch"
I looked through the list of updates, found the one that read "Fedora release files | fedora-release-15-3 (noarch)" and unchecked it. That did the trick and all other updates installed without any issues. However, error was still there when I tried to update after a reboot!
So, I looked it up on Fedora forums and apparently there is a thread here. First message is from June, so this is issue has been around for at least two months but not yet fixed. As a workaround, you can drop to a terminal window and type the following:
sudo yum update
Sounds easy enough, but as some people pointed out, this is a terrible welcome message for a newcomer to the platform. People expect things to "just work" these days and are less likely to cut Fedora a slack.
Network:
Although, there was no network connectivity until I selected my wireless, Fedora had no issues remembering and connecting to my wifi network afterwards (see this Apple thread if you are wondering where that comment come from). Yet, I think from a usability perspective, I would want OS to ask me to choose a connection upon first login if it detects a wifi connection.
Speaking of network, "Nautilus" > "Browse Network" failed to detect my QNAP samba shares but I was able to click "Go" > "Location" and access public shares by typing:
smb://myqnap/public
Power Management:
It looks like Power Management is a bit aggressive Out of Box as in OS X. If you do not use your machine for about 20secs, screen becomes darker. Fully charged laptop claimed it would drain in about 2hrs but I am yet to test how fast it discharges under my normal usage.
GUI/General:
Ubuntu has live, warm colors (I like "Orange") out of box (OOB), Fedora has "grey" as its primary color. I think it's a bad choice as it fails to give a polished look when you log in for the first time. Yes, it's of course easy to change this and some Linux-fans loath eye-candy but first-impression matters.
Gnome 3.0 is a radical change from past. It's annoying for to save something into ~/Desktop folder only to find that it does not show up on the Desktop. I know the arguments against it but we will see if this heavy-handed approach will work (I am betting it will not as it creates confusion).
Also, there is just "Log Out" option when you click your name on top right (I can follow the logic), but that means you have to log-out first and then reboot/shut-down. Well, the option to power off is actually there but hidden, so you need to use "Alt" key. Alternatively, you can hit Alt+F2 and type
reboot
Simply typing the following would not work as you must be root.
shutdown
You can hit Windows key to bring up OS X's spotlight-like search tool where you can type
terminal
and then
sudo shutdown
Tiring? Yeah! It does not really matter that much on a mobile platform as people would usually prefer to put the machine to sleep anyway, which may be one of the reasons why shutdown is not there, but seems counter-intuitive on a desktop platform.
By default, Windows only have "close" (X) button.
It's not that difficult to add them using "gconf-editor", which you must install via add/remove programs or simply by typing the following in terminal window:
sudo yum install gconf-editor
See the screenshot on the line you need to edit. Log out, log back in and you have the minimize, maximize buttons. You can even shift them from right to the left like Ubuntu by changing the location of ":" like the following:
close,minimize,maximize:
Well, these are just a couple of my first-impression notes. Ars Technica also has two good articles to read "Fedora - first Look" and "Gnome review". There is a lot of talk about the new systemd, and I am looking forward to checking it out.
2011-08-20
Your personal domain with Google Apps
So, what are the options?
Well, I guess the answer depends on what you want to do with it. There are many services out there that let you register your domain. GoDaddy.com is probably one of the most famous one. I used their service and it got better over the time. If, for example, all you want is that people reach to your blog when they type your new domain name, almost all registrars do that free of charge.
But, then what? I will tell you what I like to do with the domain names I register for personal use. Well, first thing first, I love Google services.
- I would like to have an e-mail service for my new domain, that's managed like gmail.
- I would like to create accounts for my family and sometimes friends as well and I would like them to have a common set of services (like calendar, storage area, contacts etc.)
One added benefit is that, your personal information is hidden from WHOIS directory. Normally, you get charged extra for that. This is especially useful, if you would like to have a private domain that you only use for certain (private) activities.
For example, you could buy something like "mysecretdomain.com" and configure an e-mail like "me@mysecretdomain.com". Then, use only that e-mail when your finances are involved (bank accounts, e-bay, amazon etc.).
You might also use Google docs from that domain only to keep private stuff and use your, say, gmail account for everything else. This might also help reduce the attack surface if someone is trying to steal your known account.
If you, for example, had a weak password recovery option with your gmail and someone guessed it, they still would not know about your secret domain where you keep the important stuff, which might reduce the damage... It's a wild world out there, you can never be careful enough!
Update: Added the pic on top (originally posted by Tom Anderson)
2011-08-16
Should you change your DNS?
A couple of days ago, I was talking to a friend who is running a small office in NY. He complained about how they were frequently having issues accessing web sites, the sluggishness and the inconsistencies they were experiencing.
The more we talked, the more it sounded like taking a look at their DNS servers was in order. They did not, however, have a dedicated DNS, and were using whatever DNS servers their ISP was assigning to them.
People do not need to use DNS servers their ISPs assigned to them. There are many public name servers that can be used instead. Google's Public DNS is probably the most famous one and I fully recommend it. Open DNS is another one that has been out there for a while.
Not only using public DNS may improve the speed of your browsing but it may also help you get a little extra security.
I recently came across a little (literally - just 163KB) utility called DNS Benchmark by Steve Gibson which can tell you which name servers would be the fastest for you.
As I guessed, Google's Name Servers, which I have been using for quite some time now were the fastest for me. In my case, it was not the name resolution speed that pushed me to switch to Public DNS. I hated it when my ISP intercepted mistyped domain names. If you do that, you should normally get a "404 - Page not found" error. Instead, you notice that the url you typed is put into a search web site branded by your ISP and results are shown to you.
ISPs are not really doing this out of goodwill to help you, they get money out of it. They get paid for the keywords and appropriate results show up near the top. You can read about that whole story and how that is being used for Phishing attacks here.
Once you decide to use a public DNS, I guess the easiest way to implement this would be to enter the IP addresses of the Public DNS in your Router and let it distribute them via DHCP. It's pretty easy to do.
Here is the screenshot from my LinkSys (Cisco) E3000 router.
In most cases, you can reach your router's set up by typing 192.168.1.1 into your browser and logging into it. The settings you see above are located under "Setup" > "Basic Setup".
By the way, if you are using Google Chrome, it is up to you to choose whether Google should display you some suggestions when you mistype a url or a domain name. Here are instructions to turn on or off this option. Good luck!
The more we talked, the more it sounded like taking a look at their DNS servers was in order. They did not, however, have a dedicated DNS, and were using whatever DNS servers their ISP was assigning to them.
Not only using public DNS may improve the speed of your browsing but it may also help you get a little extra security.
I recently came across a little (literally - just 163KB) utility called DNS Benchmark by Steve Gibson which can tell you which name servers would be the fastest for you.
As I guessed, Google's Name Servers, which I have been using for quite some time now were the fastest for me. In my case, it was not the name resolution speed that pushed me to switch to Public DNS. I hated it when my ISP intercepted mistyped domain names. If you do that, you should normally get a "404 - Page not found" error. Instead, you notice that the url you typed is put into a search web site branded by your ISP and results are shown to you.
ISPs are not really doing this out of goodwill to help you, they get money out of it. They get paid for the keywords and appropriate results show up near the top. You can read about that whole story and how that is being used for Phishing attacks here.
Here is the screenshot from my LinkSys (Cisco) E3000 router.
In most cases, you can reach your router's set up by typing 192.168.1.1 into your browser and logging into it. The settings you see above are located under "Setup" > "Basic Setup".
By the way, if you are using Google Chrome, it is up to you to choose whether Google should display you some suggestions when you mistype a url or a domain name. Here are instructions to turn on or off this option. Good luck!
2011-07-28
Ubuntu Unity
I've installed every version of Ubuntu released in the last four years and enjoyed it getting more and more user-friendly. I was so comfortable with it that I removed windows XP from my father-in-law's laptop and replaced it with Ubuntu about a year ago. I had to spent a couple of hours with him to set him up but that was all to it. He has been using it happily since then.. And I have the piece of mind as I am no longer worried he will be getting malware / viruses on his laptop which is connected to my home network.
At the end of April, Ubuntu released 11.04. I upgraded my laptop but left his laptop alone. Ubuntu 11.04 release brought a radical GUI change named Ubuntu-Unity. Personally, I did not find it too difficult to use although it did not seem to me that it made things any easier than they used to be.
Today, we upgraded my father-in-law's laptop to Ubuntu 11.04 as well. I let him use it for a while and it's been absolute nightmare for him. He hated the new interface, it was way too confusing for him. One of the reason for the new interface was to simplify the interface and make it easy to find things. Unfortunately, his experience was exact opposite. So, I rolled him back to Classic Ubuntu (Gnome) interface. You can find detailed instructions here to roll back.
"The new, highly simplified desktop interface “borrowed consciously” from “other successful platforms,” including Windows and Mac OS X, Shuttleworth said." according to eweek.com.
So, I will have him try OS X to see if he is going to have the same challenges. If he can use OS X just fine, then maybe these borrowed ideas were not implemented well enough in Unity.
Update [08/01/2011]: He found OS X easier to use than Unity.
2011-07-21
Simplee
Set up is easy. You have to provide your credentials to log into your healthcare provider's web, and then rest is a well designed page that you can see an overview of your healthcare situation and can drill down as necessary.
It was quite astonishing to see how high the charges are and how you are shielded (or not) from such costs.
There is some information that I am not able to explain as it claims that I owe some money because of some visits but in reality my healthcare provider covered those. Not sure if it is some kind of mistake or just a misunderstanding on my part. They have a Twitter account where you can post questions as well.
Oh, here is the LifeHacker article on it.
2011-07-03
Passwords - 2
It's been two years since I posted an entry about Passwords and highlighted an issue where even a highly respected company like Amex would only allow you to create weak passwords.
Chase too has some 'interesting' limitations on what I can use in passwords. The reason I am highlighting Amex is that their version is extreme + I love Amex! My experience with their customer service has always been quite positive. Anyway, back to subject...
Two years ago, below were the rules under which American Express 'allowed' you to create a password:
Your Password should:
* Contain 6 to 8 characters - at least one letter and one number (not case sensitive)
* Contain no spaces or special characters (e.g., &, >, *, $, @)
* Be different from your User ID and your last Password
Two years later rules have changed for better, but not by much:
Your Password:
* Must be different from your User ID
* Must contain 8 to 20 characters, including one letter and number
* May include the following characters: %,&, _, ?, #, =, -
* Your new password cannot have any spaces and will not be case sensitive.
Why on earth Amex would still insist that their customers cannot create CaSe SenSitiVe passwords is beyond me. It's a well known 'good-practice' to mix Upper and Lower case letters in passwords. There is no way security team in Amex does not know about this. So, why not allow it???
I asked them in Twitter to find out. Well, as you can see from exchange below. They won't say why..
While on subject, Steve Gibson has a fun page titled Password Haystack. Worth taking a look.
Chase too has some 'interesting' limitations on what I can use in passwords. The reason I am highlighting Amex is that their version is extreme + I love Amex! My experience with their customer service has always been quite positive. Anyway, back to subject...
Two years ago, below were the rules under which American Express 'allowed' you to create a password:
Your Password should:
* Contain 6 to 8 characters - at least one letter and one number (not case sensitive)
* Contain no spaces or special characters (e.g., &, >, *, $, @)
* Be different from your User ID and your last Password
Two years later rules have changed for better, but not by much:
Your Password:
* Must be different from your User ID
* Must contain 8 to 20 characters, including one letter and number
* May include the following characters: %,&, _, ?, #, =, -
* Your new password cannot have any spaces and will not be case sensitive.
Why on earth Amex would still insist that their customers cannot create CaSe SenSitiVe passwords is beyond me. It's a well known 'good-practice' to mix Upper and Lower case letters in passwords. There is no way security team in Amex does not know about this. So, why not allow it???
I asked them in Twitter to find out. Well, as you can see from exchange below. They won't say why..
While on subject, Steve Gibson has a fun page titled Password Haystack. Worth taking a look.
2011-06-28
2011-06-22
Running PowerShell from NAS without Prompts
It's been more than a year since I bought my QNAP TS-439 Pro NAS. Slowly I moved away from storing files locally and nowadays I store everything in there, backing up nightly to a Drobo box via front USB Interface.
Today, I wrote a powershell script that was only a couple of lines long to dump Docsis Diagnosis Info of my Modem (See DocsDiag). It simply prints verbose info on a file that has date + time stamp.
I stored the file in my "HomeShare" which I map from my NAS. When I ran the script I got the familiar Execution Policy warning that you would see if you download a script file from internet, "OR", if you run the script from a network location.
Security Warning
Run only scripts that you trust. While scripts from the Internet can be useful, this script can potentially harm your computer. Do you want to run Z:\scripts\powershell\cable\get-ModemInfo.ps1?
[D] Do not run [R] Run once [S] Suspend [?] Help (default is "D"):
If you're doing this at your office, and your devs need to run scripts from Network Location, then you would probably use GPO to set Execution Level for these scripts and use Internet Explorer GPO settings to define your network (possibly using your domain name as in *.foobar.com) as Local Intranet.
For a home user like me, however, without a domain but with network storage, solution is to make changes manually:
Update [07/02/2011]
What a difference switching to Docsis 3.0 modem did! Speed test is rarely reporting under 14Mbit/s even on busy evening hours.
Today, I wrote a powershell script that was only a couple of lines long to dump Docsis Diagnosis Info of my Modem (See DocsDiag). It simply prints verbose info on a file that has date + time stamp.
$cmd = "java -cp docsdiag.jar docsdiag -vv"
$zaman=get-date -uformat "%Y.%M.%d_%H.%M"
$dosya="modem.diags_" + $zaman + ".txt"
Invoke-Command -ScriptBlock { invoke-expression $cmd } |out-file $dosyaI stored the file in my "HomeShare" which I map from my NAS. When I ran the script I got the familiar Execution Policy warning that you would see if you download a script file from internet, "OR", if you run the script from a network location.
Security Warning
Run only scripts that you trust. While scripts from the Internet can be useful, this script can potentially harm your computer. Do you want to run Z:\scripts\powershell\cable\get-ModemInfo.ps1?
[D] Do not run [R] Run once [S] Suspend [?] Help (default is "D"):
If you're doing this at your office, and your devs need to run scripts from Network Location, then you would probably use GPO to set Execution Level for these scripts and use Internet Explorer GPO settings to define your network (possibly using your domain name as in *.foobar.com) as Local Intranet.
For a home user like me, however, without a domain but with network storage, solution is to make changes manually:
- Set Execution Policy to either RemoteSigned (recommended) or Unrestricted. For this, launch PowerShell as admin (right click start as Admin) and type
Set-ExecutionPolicy RemoteSigned or Set-ExecutionPolicy Unrestricted
I know some people may not feel easy about setting scripts to Unrestricted but I write my own scripts, do not download from Internet and run them. As long as I am on the terminal, I can bypass this check anyway.
- I defined my NAS in "Local Intranet" in Internet Explorer Zone Settings. I have several mappings from my NAS, and use both IP address and an alias when mapping drives, so I added both:
In IE, Tools > Internet Options > Security > Local Intranet > Sites, Clicked Advanced and
- added file ://qnap (qnap is an alias in my c:\Windows\System32\Drivers\Etc\Hosts file for my NAS IP)
- added 192.168.1.10 (IP address of NAS)
This did the trick. I no longer get the prompt. Funny thing is that Cablevision came by today replacing my old Cable Modem which was running Docsis 1.1 with one that's running Docsis 3.0. An then I found out that I can no longer query my modem using DocsDiag utility because apparently ISPs are now blocking SNMP.
Update [07/02/2011]
What a difference switching to Docsis 3.0 modem did! Speed test is rarely reporting under 14Mbit/s even on busy evening hours.
2011-05-15
On Hacking Windows with Scripting Tools
It's been a while since I installed my new PC and as I install more and more apps over time, the list of those that creep into system start-up increases. so, I ran ever-useful Autoruns and deleted the unnecessary ones.
It's not always easy to recognize what each executable is all about. E.g. "JMB36X IDE Setup" is the title of an app here: %windir%\RaidTool\xInsIDE.exe. At the end, it was harmless and I had it probably b/c I have a JMicron JMB36X chipset on the motherboard and vendor installed all the utils (Although I did not have a RAID config).
Anywho, sometime ago I had noticed that "gathernetworkinfo.vbs" was scheduled to run every week on my Win7 and tracked it down to a Microsoft script but while looking up its uses I found a good white-paper from Sans.org titled "Using Windows Script Host and COM to hack Windows". All that stuff is even more relevant as PowerShell is gaining in popularity and thanks to .NET PowerShell has even more power than WScript ever had.
It's not always easy to recognize what each executable is all about. E.g. "JMB36X IDE Setup" is the title of an app here: %windir%\RaidTool\xInsIDE.exe. At the end, it was harmless and I had it probably b/c I have a JMicron JMB36X chipset on the motherboard and vendor installed all the utils (Although I did not have a RAID config).
Anywho, sometime ago I had noticed that "gathernetworkinfo.vbs" was scheduled to run every week on my Win7 and tracked it down to a Microsoft script but while looking up its uses I found a good white-paper from Sans.org titled "Using Windows Script Host and COM to hack Windows". All that stuff is even more relevant as PowerShell is gaining in popularity and thanks to .NET PowerShell has even more power than WScript ever had.
2011-04-20
Nexus S Battery Awesomeness!
In my previous post, I mentioned how upset I was with Google Maps and specifically with Latitude as it was draining battery of my Nexus S. I was fully charging the phone over night and finding the battery hitting 10% in about 8 hours even if I did not use the phone at all.
OK, here is the awesome news. After 31 hours I still have 33% left on battery!!!! Just to get the screenshots of this, I installed the JDK and Android SDK (it's a shame I had to go through all that just to get a screenshot).
I updated many apps in the last couple of days so I am not really sure what exactly caused this incredible improvement but I am hoping it will stick around.
OK, here is the awesome news. After 31 hours I still have 33% left on battery!!!! Just to get the screenshots of this, I installed the JDK and Android SDK (it's a shame I had to go through all that just to get a screenshot).
I updated many apps in the last couple of days so I am not really sure what exactly caused this incredible improvement but I am hoping it will stick around.
2011-04-15
Sad State of Optimum Cable
2011-04-08
Using Latitude on Android? Bad move...
Google's Latitude has been around for a while. It only made sense for me once both my wife and I started to use Android (Google Nexus S). With Google Latitude, it was possible to keep track of each other and coordinate life events better...Unfortunately, I noticed that my battery was going down much quicker and "Battery Used" reports were pointing to "Maps". As I was not really using maps or gps, I suspected Latitude, which is part of Google Maps suite, was the cause.
I posted a question on Google Mobile forums asking if anyone else was seeing the same issue. Only one person reported the exact same issue and also suggested to log out of latitude. Tested it and confirmed that it worked well, which makes the latitude and possibly the history feature the culprit. Filed a defect here. Hopefully, they will fix it soon, I like to be able to use it again!
Subscribe to:
Posts (Atom)