Connecting to an HTTP Web Service from VBA via Proxy

Connecting to an HTTP Web Service from VBA Excel via a Proxy Server

Though MSDN suggests using stubs generated from the WSDL by MS Soap Toolkit for connecting to an HTTP Web Service from within VBA Excel, but it might not work as you would like it to, especially for a SOA-compliant web service and particularly in the cases where you need to access the service via a Proxy Server.

I have used the SOAP Connector called 'httpConnector30' successfully to connect to an HTTP Web Service without any issues. This connector is a part of the Microsoft SOAP Library named MSSOAPLib30 and you got to make sure that this library is referenced in your Excel installation. If it's not already there in your excel, just add the corresponding DLL and you're done.

Using httpConnector30 is different from consuming a Web Service by creating the stubs using MS Soap Toolkit. 'httpConnector30' requires you to specify the actual Web Service URL whereas the toolkit asks you the WSDL url and creates stubs accordingly, which you use in your VBA code. I personally think using 'httpConnector30' is easier and more straightforward if you have the service url.

Before we jump on to the code listed below, let's understand what all the code does broadly:-

• Instantiating the SOAP Connector
• Setting up the Proxy Server and Port (if access needed via Proxy)
• Setting up the Web Service URL (not WSDL url)
• Setting up Timeout period for the service call
• Setting up the SOAP Action i.e., the actual method to be called
• Beginning SOAP Message and getting connector's Input Stream
• Building up the SOAP Request (as per your Web Service definition)
• Sending the SOAP Message (this is where the service call is made)
• Initializing the SOAP Reader and reading the SOAP Response

Note: in the below code I have not shown the exception handling blocks, which you should include to grab and handle the potential errors gracefully. For example: you should first check the 'connector' as 'Not Nothing' before trying to load the 'reader' with connector's output stream.

Additionally, I've assumed that the first node (except the generic envelope and body) of the SOAP Response is actually a List and hence I've put a loop to iterate through it. 'Set response = reader.RPCResult.childNodes' actually sets the 'response' to the first node of the SOAP Response as read from the reader (which itself is loaded with the connector's output stream).

Just to make your service consumption code robust and independent of the Response Structure changes (like addition of new nodes and/or reordering of nodes), in your client code, you should iterate through all the SOAP Response nodes and compare the current node name with your Service Response node names (you can get them in the service WSDL) and subsequently handle the particular node, say inside an if-block. This will make sure that your code doesn't fail abruptly in case Service Response Structure changes. For example: it will avoid any code failure say because you had written it assuming the first node in the response was a List and let's say the service response structure changes make it the second node in the response - maybe because the service provider needed to add another field in the response and also wished to make that the first field. I know the service provide will certainly let the client developers know about the changes, but if you make your code flexible to such possible changes, nothing like it... right?

Public Sub HTTPConnectivityTest()
  
   'Instantiating the SOAP Connector
   Dim connector As New MSSOAPLib30.HttpConnector30
  
   'Setting up the Proxy Server and Port
   connector.Property("ProxyServer") = "fully-qualified-proxy-server-or-IPAddress:Port"

  
   'Setting up the Web Service URL
   connector.Property("EndPointURL") = "http://web-service-server:port/webservices/SampleService.v1"
  
   'Setting up Timeout period for the service call
   connector.Property("Timeout") = 2000 '2 minutes
  
   'Setting up the SOAP Action i.e., the actual method to be called
   connector.Property("SoapAction") = "urn:getSampleData"
  
   'Beginning SOAP Message
   connector.BeginMessage
  
   'Initializing SOAP Serializer with connector's input stream
   Dim writer As New MSSOAPLib30.SoapSerializer30
   writer.Init connector.InputStream

   'Building the SOAP Request - envelope and body
   writer.startEnvelope              ' <SOAP-ENV:Envelope>
   writer.startBody                  ' <SOAP-ENV:Body>

   'Populating the SOAP Request with actual input parameters
   writer.startElement "SampleServiceRequest", "service namespace", , "s3"   ' <SampleServiceRequest>
  
   writer.startElement "inputParam1"   ' <inputParam1>
   writer.writeString "param1 value"    ' value of inputParam1
   writer.endElement                 ' </inputParam1>

   writer.startElement "inputParam2"   ' <inputParam2>
   writer.writeString "param2 value"        ' value of inputParam2
   writer.endElement                 ' </inputParam2>

   writer.startElement "inputParam3"   ' <inputParam3>
   writer.writeString "param3 value"        ' value of inputParam3
   writer.endElement                 ' </inputParam3>

   'Populating list-type parameter
   writer.startElement "paramList"   ' <paramList>

    'Adding node #1 to the list-type param
    writer.startElement "paramListNode"    ' <paramListNode>
     writer.startElement "nodeParam1"          ' <nodeParam1>
      writer.writeString "value1"                    ' value of nodeParam1
     writer.endElement                          ' </nodeParam1>
  
     writer.startElement "nodeParam2"          ' <nodeParam2>
      writer.writeString "value1"                    ' value of nodeParam2
     writer.endElement                          ' </nodeParam2>
    writer.endElement                          ' </paramListNode>
 
    'Adding node #2 to the list-type param
    writer.startElement "paramListNode"    ' <paramListNode>
     writer.startElement "nodeParam1"          ' <nodeParam1>
      writer.writeString "value2"                    ' value of nodeParam1
     writer.endElement                          ' </nodeParam1>
  
     writer.startElement "nodeParam2"          ' <nodeParam2>
      writer.writeString "value2"                    ' value of nodeParam2
     writer.endElement                          ' </nodeParam2>
    writer.endElement                          ' </paramListNode>
  
   'Population of list-type param ends here
   writer.endElement                          ' </paramList>

   'Finishing the SOAP Request
   writer.endElement                 ' </SampleServiceRequest>
   writer.endBody                    ' </SOAP-ENV:Body>
   writer.endEnvelope                ' </SOAP-ENV:Envelope>
  
   'Sending the SOAP Message (this is where the service call is made)
   connector.EndMessage
  
   'Defining SOAP Reader and initializing it with connector's output stream
   Dim reader As New MSSOAPLib30.SoapReader30
   reader.Load connector.OutputStream
  
   'Parsing the SOAP Response
   Dim response As MSXML2.IXMLDOMNodeList

   'Setting the response to the first node of the SOAP Response
   Set response = reader.RPCResult.childNodes
   Dim node As MSXML2.IXMLDOMNode

   'Iterating through the first node of SOAP Response knowing it is a list
   For Each node In response
    Dim nodeName As String
    Dim nodeValue As String
  
    nodeName = node.nodeName
    nodeValue = node.nodeTypedValue
  
    'Showing the Node Name and Value on Alert Boxes 
    MsgBox node.nodeName & ": " & node.nodeTypedValue
   Next node
End Sub

Liked the article? Subscribe to this blog for regular updates. Wanna follow it to tell the world that you enjoy GeekExplains? Please find the 'Followers' widget in the rightmost sidebar.

Passing '\n' (new-line) on command line in Java

Can we pass a new-line ('\n') character or any other escape sequence via command line in Java?

One of our visitors (Vivek Athalye) asked this in response to the article - Tricky use of static initializer block. Thought of posting the answer as a separate article to increase the chances of it reaching to a wider range of audience.

The answer to the query is NO. The question arises, if you pass the same escape sequence programmtically, it works fine, so why doesn't it work well when passed via command line? For example: System.setProperty("line.separator", " Bye!\nBBye!"); will work fine, but if try to do the same via command line as (java -Dline.separator=" Bye!\nBBye!" ClassName) then '\n' will be treated as two distinct ASCII characters ('\' and 'n') and not as a single escape sequence new-line Unicode character.

This behavior was logged as a bug on Sun's Bug Database on Oct 31, 2003. But, it was closed saying 'not a bug' on Nov 05, 2003. The reason given is that interpretation of text passed on command line is a shell specific stuff and it is not reasonable to expect that to work in lines with the handling of escape sequences by any particular programming language.

It's not something to do with Java as even if you pass a command line argument having '\n' to a C program, it will be treated as two distinct ASCII characters only and not as a escape sequence.

What stops a shell to interpret escape sequences is that escape sequences are represented differently in different programming languages - like '\n' is actually a single character Unicode character whereas in C it's a two-character ASCII sequence having a different meaning because of the preceding '\' character. So, on a system which requires to run both Java and C programs, which convention should the shell use? Tomorrow, if we see any other representation of escape sequence by some other programming language, how will the already developed shell will cope up with that? Hence, shell plays it straight and simply passes everything written on command line as ASCII character sequences without giving any special meaning to any particular sequence. Fair enough, I believe.

Liked the article? Subscribe to this blog for regular updates. Wanna follow it to tell the world that you enjoy GeekExplains? Please find the 'Followers' widget in the rightmost sidebar.

Binary rep of negative numbers in Java - 2's complement

2's Complement: Binary representation of negative numbers in Java

Negative numbers in Java are represented using 2's complement. As we know that integers in Java occupy 4 bytes so to understand how a negative integer (say -4) is represented internally in Java, we first need to find the binary equivalent of the positive value of the integer (in this case 4) and subsequently by finding the 2's complement of that binary representation.

Okay, so how do find 2's complement of a binary number? Simply by adding '1' to the 1's complement of that number. But, how to find 1's complement of a binary number then? Just by reverting the bits of the number i.e., changing 1s to 0s and 0s to 1s. An example may of of some help here.

...
int i = -4;
...

Step #1: Binary Equivalent of the positive value (4 in this case)

0000 0000 0000 0000 0000 0000 0000 0100

Step #2: 1's complement of the binary rep of 4 by inverting the bits

1111 1111 1111 1111 1111 1111 1111 1011

Step #3: Finding 2's complement by adding 1 to the corresponding 1's complement

1111 1111 1111 1111 1111 1111 1111 1011
0000 0000 0000 0000 0000 0000 0000 0001
---------------------------------------
1111 1111 1111 1111 1111 1111 1111 1100

Thus, we see that integer -4 is represented by the binary sequence (1111 1111 1111 1111 1111 1111 1111 1100) in Java.

Once we have an understanding of how the numbers are represented internally, bit-level manipulation becomes easily understandable, which otherwise is obviously one of the hardest things in Java (or any other language supporting that) to visualize.

Liked the article? Subscribe to this blog for regular updates. Wanna follow it to tell the world that you enjoy GeekExplains? Please find the 'Followers' widget in the rightmost sidebar.

Viewing/Editing PPT, Doc, TIFF, etc. in your browser

Viewing/Editing files without having the required s/w or tools installed

Wonder what will you do in case you end up getting caught in a situation where none of the widely used tools/softwares (such as MS Office, Flash, File Viewers/Editors like Acrobat, etc.) are installed on a machine on which you need to at least view (or maybe edit if possible) some docs/files?

Well... until recently it was simply not possible, but the release of Google Docs has made it possible for many file types including PDF, Doc, etc. and now also for PowerPoint and TIFF. Now you only need a browser and an Internet connection and you can easily View/Print (and also Edit some of types) most of these popular file types.

This all can be done from your Gmail account straightaway. PDF Viewing was enabled way back in Dec 2008 and the most recent release from Google in this regard is PPT and TIFF file viewing.

'View as slideshow' option was there for PPTs, but now they have integrated the conversion technology into the same viewer which they are using for PDFs and TIFFs. Moreover, the new viewer supports a richer set of features like zoom in/out, printing PPT to a PDF doc, etc. Additionally, you no longer require a Flash plug-in installed on the machine. Below is the sample screenshot showing you view PPTs in the Gmail Viewer.


Most of the default TIFF viewers show only the first page, but the online viewer will not only show you all pages, but also give you an option to print the TIFF file to a PDF doc. Read the official Gmail blog post on this here.

Google Labs has already provided so much to be used for free and the list is still growing. For more details, visit the official Google Labs page, if not visited already.

Liked the article? Subscribe to this blog for regular updates. Wanna follow it to tell the world that you enjoy GeekExplains? Please find the 'Followers' widget in the rightmost sidebar.

SAX v/s DOM. How to choose between DOM and SAX?

Differences between DOM and SAX. When to use what?

Before going through the differences, if you need a refresh of what SAX and DOM are, please refer to this article - SAX, DOM, JAXP, & JDOM >>.

While comparing two entities, we tend to see both of them as competitors and consequently comparing them to find a winner. This of course is not applicable in every case - not at least in the case of SAX and DOM. Both have their own pros and cons and they are certainly not in direct competition with each other.

SAX v/s DOM

Main differences between SAX and DOM, which are the two most popular APIs for processing XML documents in Java, are:-

• Read v/s Read/Write: SAX can be used only for reading XML documents and not for the manipulation of the underlying XML data whereas DOM can be used for both read and write of the data in an XML document.
• Sequential Access v/s Random Access: SAX can be used only for a sequential processing of an XML document whereas DOM can be used for a random processing of XML docs. So what to do if you want a random access to the underlying XML data while using SAX? You got to store and manage that information so that you can retrieve it when you need.
• Call back v/s Tree: SAX uses call back mechanism and uses event-streams to read chunks of XML data into the memory in a sequential manner whereas DOM uses a tree representation of the underlying XML document and facilitates random access/manipulation of the underlying XML data.
• XML-Dev mailing list v/s W3C: SAX was developed by the XML-Dev mailing list whereas DOM was developed by W3C (World Wide Web Consortium).
• Information Set: SAX doesn't retain all the info of the underlying XML document such as comments whereas DOM retains almost all the info. New versions of SAX are trying to extend their coverage of information.

Usual Misconceptions

• SAX is always faster: this is a very common misunderstanding and one should be aware that SAX may not always be faster because it might not enjoy the storage-size advantage in every case due to the cost of call backs depending upon the particular situation, SAX is being used in.
• DOM always keeps the whole XML doc in memory: it's not always true. DOM implementations not only vary in their code size and performance, but also in their memory requirements and few of them don't keep the entire XML doc in memory all the time. Otherwise, processing/manipulation of very large XML docs may virtually become impossible using DOM, which is of course not the case.

How to choose one between the two?

It primarily depends upon the requirement. If the underlying XML data requires manipulation then almost always DOM will be used as SAX doesn't allow that. Similarly if the nature of access is random (for example, if you need contextual info at every stage) then DOM will be the way to go in most of the cases. But, if the XML document is only required to be read and that too sequentially, then SAX will probably be a better alternative in most of the cases. SAX was developed mainly for pasring XML documents and it's certainly good at it. SO, if you need to process an XML document maybe to update a datasource, SAX will probably make a alternative.

Requirements may certainly fall between the two extremes discussed above and for any such situation you should weight both the alternatives before picking any of the two. There are applications where a combination of both SAX and DOM are used for XML processing so that might also be an alternative in your case. But, basically it would be a design decision and evidently it would require a thorough analysis of the pros and cons of all possible approaches in that situation.

Read Next: A step-by-step implementation (with explanation of the code) of a SAX Parser in Java using SAX2 APIs - Simple SAX Parser Impl in Java >>

Liked the article? Subscribe to this blog for regular updates. Wanna follow it to tell the world that you enjoy GeekExplains? Please find the 'Followers' widget in the rightmost sidebar.

Sax, DOM, JAXP, & JDOM. Evolution of Java-XML combo.

Evolution of the XML Parsing/Manipulation using Java

The combination of Java and XML has been one of the most attracting things which had happened in the field of software development in the 21st century. It has been mainly for two reasons - Java, arguably the most widely used programming language and XML, almost unarguably the best mechanism of data description and transfer.

Since these two were different technologies and hence it initially required a developer to have a sound understanding of both of these before he can make the best use of the combination. Since then there have been a paradigm shift towards Java and we have seen few interesting technologies getting evolved to make this happen. Some of them are:-

SAX - Simple API for XML Parsing

It was the first to come on the scene and interestingly it was developed in the XML-Dev maling list. Evidently the people who developed this were XML gurus and it is quite visible in the usage of this API. You got to have a fair understanding of XML, but at least Java developers got something to combine the two worlds - Java and XML in a structured way. It instantly became a hit for the obvious reasons.

Being the first in the evolution ladder, it obviously had only the basic support for XML processing. It is an event-based technology, which uses callbacks to load the parts of the XML document in a sequential way. This effectively means you can't go back to some part which was read/processed previously - if you do have such a requirement then you would need to store/manage the relevant data yourself.

Since this API does require to load the entire XML doc and also because it offers only a sequential processing of the doc hence it is quite fast. Another reason of it being faster is that it does not allow modification of the underlying XML data.

Interested in going through a step-by-step implementation (with explanation of the complete source code) of a simple SAX Parser in Java using SAX2 APIs? Here is it for you - SAX Parser Implementation in Java >>

DOM - Document Object Model

The Java binding for DOM provided a tree-based representation of the XML documents - allowing random access and modification of the underlying XML data. Not very difficult to deduce that it would be slower as compared to SAX.

The event-based callback methodology was replaced by an object-oriented in-memory representation of the XML documents. Though, it differs from one implementation to another if the entire document or a part of it would be kept in the memory at a particular instant, but the Java developers are kept out of all the hassle and they get the entire tree readily available whenever they wish.

JAXP - Java API for XML Parsing

The creators and designers of Java realized that the Java developers should not be XML gurus to use the XML in Java applications. The first step towards making this possible was the evolution of JAXP, which made it easier to obtain either a DOM Document or a SAX-compliant parser via a factory class. This reduced the dependence of Java developers over the numerous vendors supplying the parsers of either type. Additionally, JAXP made sure that an interchange between the parsers required minimal code changes.

JDOM - Java Document Object Model

Even though JAXP reduced the need for caring about the different parser implemenattions, still it required the developers to use either the DOM or SAX for manipulating the XML data. JDOM evolved as the designers of Java APIs thought of moving more towards Java and Java-like constructs while processing XML documents and it supported moving away from non-Java structs like Attributes (in SAX) and NamedNodeMap (in DOM). Now the Java developers can use the mucm more familiar Java Collection classes to manipulate XML data. Moving towards the customary Java constructs also helped making the processing faster - almost at par with SAX.

So, now that we are aware of what SAX and DOM are, let's move towards discussing the differences between the two. As is the case with most of the other technological comparisons, neither of the two is an absolute favourite and the choice would more often than not depend upon your requirement. SAX v/s DOM. When to use what?

Liked the article? Subscribe to this blog for regular updates. Wanna follow it to tell the world that you enjoy GeekExplains? Please find the 'Followers' widget in the rightmost sidebar.

How to recover Shift + Deleted folders/emails in Outlook?

How to recover Shift + Deleted folders/emails in Microsoft Outlook?

Ever wondered if you can really recover something in Windows, which you have Shift + Deleted? Well, you can do that at least in few cases (if not in all) - one of them being recovering folders/emails which you accidentally Shift + Deleted in your Microsoft Outlook (running with MS Exchange Server). I managed to delete one important folder this weekend and only then I could realize the importance of this feature.

As you would certainly be aware that a plain 'Delete' moves the particular email to the 'Deleted Items' folder, from where you can easily restore them unless they get permanently deleted from that folder once one of the two situations occur - either the Deleted Items folder exceeds its quota of storage or the deleted emails become stale more than the allowed number of days (which I guess is 20/30 days) - whichever happens earlier.

But, it's interesting to know that even a Shift + Delete, does not really instantly delete all the items permanently - not at least in case of Microsoft Outlook. It's just that you probably can't see an option to recover them with your default settings. You need to add one registry entry and that's it. They will be visible to you then and you can recover the same way as you restore deleted items from the 'Deleted Items' folder. How many of these emails, folders, or emails within the folders you can recover, again depends upon the two parameters - storage quota and the age of the Shift + Deleted emails. So, you probably need to act fast whenever you manage to lose your folders/emails accidently. Find below the steps which are required for this recovery:-

Registry Entry: if not already there (it's required to be added only once after the installation of the client application), you would be required to add the below entry in your Windows Registry. Below are the steps:-

• Run -> regedit
• expand HKEY_LOCAL_MACHINE
• expand and go to SOFTWARE
• expand and go to Microsoft
• expand and go to Exchange
• expand and go to Options
• add the entry 'DumpsterAlwaysOn REG_DWORD 0x00000001 (1)'

Microsoft Outlook -> Tools -> Recover Deleted Items...: If you want to recover a folder then simply select the corresponding mailbox and select 'Recover Deleted Items...', which will enable you select and recover the Shift + Deleted folders of that mailbox. If you want to recover emails then simply select the corresponding folder and do the same. It will show you all the emails of that folder available for you to be recovered (based on storage quota and age of the emails as discussed above). Please find below the sample screenshots for your reference:



Liked the article? Subscribe to this blog for regular updates. Wanna follow it to tell the world that you enjoy GeekExplains? Please find the 'Followers' widget in the rightmost sidebar.

Outlook Express v/s Outlook. Which suits you better?

Outlook Express v/s MS Outlook. How to choose one between the two?

Outlook Express v/s Microsoft Outlook

Many of you would already be aware of the main difference between the two. For those, who might not have cared to thought of it so far, here is the summary of the main difference(s) between the two messaging and collaboration clients delivered to us by Microsoft:-

Outlook Express: it has been designed mainly for Home Users to provide them a reliable and fast email and newsgroup functionality. It can use any Internet standard system, for example, SMTP, POP3, and IMAP. It has full support for you to take advantage of the technologies like LDAP, MHTML, HTML, S/MIME, and NNTP. In addition, you get to use many useful features including the ability to receive mail from multiple e-mail accounts, the ability to create Inbox rules, etc. The support for HTML enables you to personalize your custom backgrounds with graphics and colors. Outlook Express also includes stationery from Greetings Workshop and Hallmark for many occasions including birthdays or special holidays.

Microsoft Outlook: this on the other hand has been designed for Business Users who require not only reliability and ease of use, but in addition also require many more e-mail functionality and a tighter integration between e-mail and tools for information management and collaboration. This stand-alone client application which is integrated into Microsoft Office and Exchange Server, provides a complete integration of e-mail, calendaring, contact management, and a seamless integration with MS Office applications. Organizing your e-mails is quite easy by using the powerful Inbox rules. When used with Exchange Server, Outlook can be used for workgroup information sharing and workflow communications, group scheduling, public folders, forms, and an enhanced Internet connectivity. In addition to the technologies supported by Outlook Express, MS Outlook also supports even more advanced technologies like vCalendar, vCard, iCalendar, MAPI, and HTML mail.

In addition, Microsoft Outlook also offers easy migration from other e-mail clients like Microsoft Mail, Microsoft Schedule+ 1.0, Lotus Organizer, NetManage ECCO, Symantec ACT, etc. It also facilitates synchronization with leading Personal Digital Assistants (PDAs), such as the 3Com Palm Pilot.

How to choose which one of the two suits you better?

Above discussion clearly shows that the Outlook Express is mainly for Home Users, whereas Microsoft Outlook is a full-fledged, reliable, and performant messaging and collaboration application for Business Users. Here Home Users doesn't necessarily mean that Outlook Express won't suit any businesses. It should be good enough for most of the Very Small Businesses, which don't require the advanced features supported by Microsoft Outlook. MSDN lists down the parameters quite clearly on the basis which one can easily decide which one of the two, one should go for. Here is what MSDN says in this regard:-

Choose Outlook Express if:

• You require only Internet e-mail and newsgroup functionality (for versions of Windows later than Microsoft Windows 95, versions of Windows earlier than Microsoft Windows 95, Macintosh, and UNIX platforms).
• You use or plan to use Office 98 for Macintosh, and you want to take advantage of the integration of Outlook Express with this version of the Office suite.

Choose Outlook if:

• You require advanced Internet standards-based e-mail and discussion group functionality.
• You require integrated personal calendars, group scheduling, task, and contact management.
• You require integrated e-mail and calendaring, cross-platform clients for versions of Windows later than Microsoft Windows 95, versions of Windows earlier than Microsoft Windows 95, and Macintosh platforms.
• You use, or plan to use Office 97, Office 2000, Office XP or Exchange Server and want to take advantage of the integration of Outlook with this version of the Office suite, and the integration with Exchange Server.
• You require robust, integrated run-time and design-time collaboration capabilities.

Read Next: How to recover Shift + Deleted folders/emails from Microsoft Outlook?

Liked the article? Subscribe to this blog for regular updates. Wanna follow the blog to manage to tell the world that you enjoy GeekExplains? Please find the 'Followers' widget in the rightmost sidebar.

LoadLibrary failed - a DLL initialization routine failed

Resolving 'LoadLibrary failed - a DLL initialization routine failed' error


There might be other reasons for this error as well, but the most common one is probably because of a missing dependent DLL. While trying to register SOAPIS30.dll (for those who are new to DLL installation - that's done by using RegSvr32.exe OR Gacutil.exe), I encountered this error.

Identifying the problem and fixing that - using Dependency Walker

As we commonly do with other DLL registration errors, I started with analyzing the DLL and trying to register it from within Dependency Walker tool. While trying to register, it straightaway gave me three errors, all indicating the same missing dependent DLL. It actually tried to look for this DLL in three directories and hence three errors for the same missing DLL.

LoadLibraryExA("C:\Program Files\Common Files\MSSoap\Binaries\Resources\2057\MSSOAPR3.DLL", 0x00000000, LOAD_WITH_ALTERED_SEARCH_PATH) returned NULL. Error: The specified module could not be found (126).

LoadLibraryExA("C:\Program Files\Common Files\MSSoap\Binaries\Resources\1033\MSSOAPR3.DLL", 0x00000000, LOAD_WITH_ALTERED_SEARCH_PATH) returned NULL. Error: The specified module could not be found (126).

LoadLibraryExA("C:\Program Files\Common Files\MSSoap\Binaries\MSSOAPR3.DLL", 0x00000000, LOAD_WITH_ALTERED_SEARCH_PATH) returned NULL. Error: The specified module could not be found (126).

How to register DLLs from within Dependency Walker?

Open the RegSvr32.exe (found in WINDOWS\System32 folder) in DW and then run the profiler by selecting the corresponding main-menu option. It will prompt you to browse and enter the particular DLL, which you want to install.

If you have corrected the error i.e., in this case if you have placed the missing DLL in any of the three directories where it is expected to be (you can easily identify the directories in the DW error messages pasted above) and assuming that everything else is fine, like the DLL is not a corrupted one, etc. then you would see the below alert showing that the registration of the DLL has completed successfully.


This alert marks the end of a successful DLL registration as is quite evident from the message displayed in the alert box.

Liked the article? Subscribe to this blog for regular updates. You may like to follow the blog to manage the bookmark easily and to tell the world that you enjoy GeekExplains. Interested? Find the 'Followers' widget in the rightmost sidebar.

The 25 Most Dangerous Programming Errors

25 Most Dangerous Programming Errors listed by CWE/SANS

The US National Security Agency has helped building a list of the top 25 world's most dangerous programming errors which may lead to security vulnerabilities that can be attacked by the cyber criminals and this list of 25 errors is now assumed to be the minimum set of coding errors that must be fixed completely before letting the code to go live.

SANS Institute (based in Maryland) believes that just two of these 25 errors led to more than 1.5 million web-site security breaches during the year 2008. One can easily imagine how large the total number of security breaches all the 25 most dreaded errors may lead to. This is the reason why the industry felt a need for having a common consolidated list of the probable errors which the programmers should refer to as a minimum set of errors they need to be extremely careful about. It's assumed that this is the first time has reached to an agreement. More than 30 organisations (including various government, corporate, and educational institutions like US National Security Agency, US Department of Energy, Microsoft, Symantec, McAfee, Purdue, etc.) have participated in this collaborative effort. CWE (Common Weakness Enumeration) and SANS Institute together issued the list on January 12, 2009.

The list of the most dreaded 25 programming errors have been listed under three categories - Insecure Interaction Between Components, Risky Resource Management, and Porous Defences. The first two categories contain 9 errors each whereas the third category contains 7 errors. The official list of The Top 25 Most Dangerous Programming Errors, listed category-wise is:-

Insecure Interaction Between Components

• CWE-20:Improper Input Validation
• CWE-116:Improper Encoding or Escaping of Output
• CWE-89:Failure to Preserve SQL Query Structure
• CWE-79:Failure to Preserve Web Page Structure
• CWE-78:Failure to Preserve OS Command Structure
• CWE-319:Cleartext Transmission of Sensitive Information
• CWE-352:Cross-Site Request Forgery
• CWE-362:Race Condition
• CWE-209:Error Message Information Leak

Risky Resource Management 

• CWE-119:Failure to Constrain Operations within the Bounds of a Memory Buffer
• CWE-642:External Control of Critical State Data
• CWE-73:External Control of File Name or Path
• CWE-426:Untrusted Search Path
• CWE-94:Failure to Control Generation of Code
• CWE-494:Download of Code Without Integrity Check
• CWE-404:Improper Resource Shutdown or Release
• CWE-665:Improper Initialization
• CWE-682:Incorrect Calculation

Porous Defenses 

• CWE-285:Improper Access Control
• CWE-327:Use of a Broken or Risky Cryptographic Algorithm
• CWE-259:Hard-Coded Password
• CWE-732:Insecure Permission Assignment for Critical Resource
• CWE-330:Use of Insufficiently Random Values
• CWE-250:Execution with Unnecessary Privileges
• CWE-602:Client-Side Enforcement of Server-Side Security

The errors may look quite trivial, but it's not-so-easy to avoid them creeping into an application of a considerable size, especially when a large application is being developed by a team(s) of sizeable people and hence it requires a well-defined process of code-review (peer and self both), checklist verification, etc. to be in place throughout the development phase. Should you require an explanation of what the above listed errors mean (which you probably won't require for most of them) then explore this link - CWE/SANS Top 25 errors list. . BBC News report on this list can be found here.

Liked the article? Subscribe to this blog for regular updates. You may also like to follow the blog to manage the bookmark easily and to tell the world that you enjoy GeekExplains. If interested then please find the 'Followers' widget in the rightmost sidebar.

XML Schema and its anatomy, Uses of XSDs, Namespaces

XML Schema and its anatomy, Applications of XML Schemas, Namespaces

XML Schema was approved as an official recommendation by the WOrld Wide Web Consortium (W3C) in 2001. W3C defines XML Schema as "XML Schemas express shared vocabularies and allow machines to carry out rules made by people. They provide a means for defining the structure, content, and semantics of XML documents."

As it's clear from the definition that XML Schema is a mechanism of defining basically the structure, content and semantics of XML Documents which conform to the Schema under consideration. But, DTD (Document Type Definition) also serves a similar purpose. So, why a need for something different was felt?

Well... the short answer is that XML Schema just evolved from DTDs in the wake of getting a more powerful and flexible mechanism as compared to what DTD supported. XML Schemas are XML documents themselves and hence all the benefits of XML - parsing, programmatically accessing, validating, and extending automatically apply to XML Schemas as well. These all benefits make Schemas a far better alternative to DTDs.

Do we still need DTDs?

Yeah, we need them, but probably only for the legacy applications where they have been in use since long. Almost all the newer applications now use Schemas only for the simple reason that XML Schemas are much more powerful & flexible and therefore offer many advantages over DTDs. However, DTDs are still supported and they can be used in tandem with XML Schemas as well.

Anatomy of XML Schemas

An XML Schema is made up of the following declarations and definitions. Few of these might be optional as well.

• Document Type Declaration: Since an XML Schema is also an XML Document which obviously conforms to the W3C XML recommendations and hence they may contain the particular document type declaration, but this is not a mandatory requirement. If present, it's inferred by the root element named <Schema>
• Namespace Declaration: Again an optional declaration which is used to provide a context for element names and attribute names used within an XML Document (which conforms to this Schema). This helps in avoiding ambiguity in name resolution and thereby helps building and extending XML Documents using URIs ensuring unique names for elements and attributes. Namespaces can either be defined inline (called inline xml namespace which are defined inline with element and atribute in the XML) or as Expanded names where the namespace name is combined with the local name to uniquely identify the particular name. Both the namespace and the local name are separated with a colon (e.g., NamespaceName:LocalName).
• Type Definitions: These definitions are used to define Simple or Complex data types or structures, which are later re-used by all the client content models.
• Element/Attribute Declarations: This section of an XML Schema defines the elements and their respective attributes which are used for tags in the XML instances using the Schema. Various constraints like id, type, max/minOccurs, substitutionGroup, etc. can also be defined in this section of an XML Schema.
• Sequence Definition: As the name might suggest this section is used to enforce the order in which the child elements of the XML instances (using the XML Schema in consideration) are required to appear.

XML Schema applications/uses

• Data Validation: XML Schema is used to define the structure of the elements and the attributes within the elements and this definition helps the XML parsers to validate the XML Document Syntax, Datatypes used by the XML Document/Instance, and/or the Inclusion of mandatory elements or attributes. This obviously helps the application designers to delegate some of the basic data validation (and data sufficiency) tasks to the XML Schema rather than doing all of them programmatically.
• Content Model Definition: XML Schema supports both simple and complex data types definitions which ultimately provides flexibility of using concepts like inheritance to the data syntax. This consequently helps building the XML Schema defining extensible models highly suitable for large and complex applications.
• Data Exchange/Integration: Since XML Schemas are themselves XML documents and hence they can be parsed and accessed similar to other XML instances by variety of XML companion tools for various purposes. For example: an XSD when used in conjunction with an appropriate XSLT and an XML-enabled database can support the changes made to the global elements defined by the XSD to be processed consistently. In addition, the output can simultaneously be produced in various formats like PDF, Doc, RTF, HTML, etc. using the single source publishing methodology. The data-oriented datatypes provided in XML Schema 1.1, in addition to the document-oriented datatypes as supported in the previous versions, facilitate complex document exchange and data integration scenarios. Namespaces supported by XML Schema can be used to have more than one vocabulary at a time in an XML instance as the namespaces enable the XML documents to contain unique identifiers. Namespaces facilitate ample opportunities for data exchange and integration by enabling the entire XML frameworks to co-exist within the same architecture. This feature is extremely helpful in mergers and acquisitions, and supply chain requirements, where we generally have a plethora of heterogeneous data constructs.
• Industry XML Standards: These standards aim to streamline and provide a basis for industry-wide data integration by implementing common XML vocabularies which enable the business partners to seamlessly exchange data across different systems and architectures. Several new industry standards are strongly being followed and they are paving way for a seamless data integration and inter-operability. Some of these standards are DITA, DocBook, SCORM, ACORD, CXML, FIXML, and XBRL.

Liked the article? You may like to Subscribe to this blog for regular updates. You may also like to follow the blog to manage the bookmark easily and to tell the world that you enjoy GeekExplains. You can find the 'Followers' widget in the rightmost sidebar.

www v/s www1 v/s www2. Role of CNAME and DNS.

How are www, www1, www2, etc. different? Role of CNAME & DNS?

Let's start with discussing what the different parts of a typical web site address mean? Say for example, if we take http://www.google.com then the first part from left http denotes the protocol being used for communication on Internet. The two most popular protocols being used are http and ftp. Requests using different protocols connect to server at a separate (default unless otherwise explicitly specified) port on the targeted server. Like an http request connects to the server via TCP and connects by default at port number 80. An 'http' request is also called a Web request.

Now that we have kind of understood what the first part of a typical URL means, let's move on to the next parts. We're discussing only the web site addresses in this article so the first part will always be either 'http' or 'https' which is nothing but HTTP running on SSL (Secure Sockets Layer). Following this there will be a set of 3 characters (://), usually termed as delimiters and they separate the protocol from the web site address.

The web site address in our example is www.google.com like every other web site address this is also of the form CNAME.secondaryName.TopLevelDomain where TopLevelDomain is something like '.com', '.org', ... etc. and they are resolved by the top level domain name servers called the root servers. These root servers are maintained by naming authroities like ICANN. The TopLevelDomain are normally one word long only, but they can be made of two (or maybe more) words separated by '.' as well, for example 'co.in', 'co.uk', ... etc. they are also top level domain names.

Left to the top level domain names comes the secondaryName which is the normally the unique name you choose for your website. Like in our example this name is 'google'. These names are resolved using the lower level domain name servers (DNS). Again it's not necessary that the secondaryName will always be one word long. It can be made up of severalwords maybe to maintain a hierarchy for the purpose of segregating (and then serving) requests of different types based on country, sub-domains, etc.

The leftmost part of a web site address is normally 'www' which is nothing but the name of the web server of the company hosting and running the web site. As it's a name and hence it can anything like 'aaa', 'aa', 'aaaa', 'www1' , 'www2' or any other string unique in context of the company specific DNS as these names are not resolved by the root or intermediate level DNS instead it's the company specific DNS which resolves it. Resolving a name simply means returning the IP address of the computer so that a combination of IP Adrress and a port (normally a default port) can be used by a client to establish a TCP connection.

This leftmost part of the web site which is normally resolved by the company specific DNS is usually called CNAME or Canonical Name. In case of our example, CNAME is 'www'. Like every other part this part is not required to be a one-word long name only. It all depends how much string is mapped to the actual IP address of the server in the company specific DNS mapping table. Needless to re-iterate that this can be any string (unique in the particular DNS context) you want and not only 'www'.

Let's take another example. The address of this blog is http://geekexplains.blogspot.com, so for this address 'com' is the top level domain, 'blogspot' is the secondaryName and 'geekexplains' is the canonical name which identifies this blog uniquely under the context of the secondary domain named 'blogspot'. The same physical server might (or probably for sure) is used for many other blogs running under 'blogspot'. This is probably achieved by having separate active ports (on the physical machine) each of which hosting processes responsible for one blog each. Ever wondered why the address 'http://www.geekexplains.blogspot.com' also takes you to the same blog? Well... because two canonical names 'geekexplains' and 'www.geekexplains' are probably mapped to the same IP Address - Port combination which hosts the process servicing the GeekExplains blog. The DNS at secondary domain requires that all the canonical names are unique in its context which is why you're asked to pick an unique name for your blog while you create one. Another interesting point to note here is that the client browser establishes TCP connection on the default Port Number 80 only as it connects to the server hosting 'blogspot.com' only and the request is further resolved there to return the HTTP response of the actual blog to the client. The mapping table of 'blogspot' DNS makes sure that the requests to any of the blogs running under it are intercepted by it and subsequently all the requests are serviced as expected. Hope this helps. Any doubts? Or, have anything to add/modify here? Feel free to reach me by dropping a comment.

Liked the article? You may like to Subscribe to this blog for regular updates. You may also like to follow the blog to manage the bookmark easily and to tell the world that you enjoy GeekExplains. You can find the 'Followers' widget in the rightmost sidebar.

How does Password Encryption work in real world?

Password encryption in work: illustration using SBI Sign In process

Last week I received an email (from one of our visitors, Anil) inquiring about what all actually takes place to ensure that the password (or any other sensitive data for that matter) gets encrypted before the request is sent to the Web/App Server? Thanks Anil for raising such a nice point.

In this article, I'll try to discuss how the password encryption feature typically works. Few details might be implementation dependent and hence might be little different in your case than what is mentioned below (and in the follow-up article), but the underlying idea will probably remain the same (more or less) for most of real world applications which require authentication.

Okay, let's start from thinking about which all places do we actually need to put encryption into action and how do we implement them? Except the possible encryption done at the Database end, there are two popular approaches of implementing encryption - One, which is done at the client side (the one we will mainly talk about in this article) and Two, which is done at the server side (i.e., the request carries the actual password and at the server it's encrypted to be processed further).

The former of the two is obviously safer to have as it eliminates the risk of the request being intercepted in the middle before it actually reaches the web/app server. Well... you can say that the data packaged in a HTTP POST request is automatically encrypted in case of HTTPS, but an extra level of encryption will only add to the security of the web application. Of course, the implementation should not be too much time consuming otherwise the benefits of having a more secure application will be ruled over by the frustration it might cause to its end-users.

Though, it depends upon the actual implementation, but possibly the preferred choice (in highly secure systems) is that the actual password should not be exposed anywhere in system, which means the encrypted password stored in DB is fetched and probably not decrypted back to actual password which the end-user uses, but instead some other form which is matched with the decrypted one at the middle-tier to authenticate the user. Find below a pictorial representation of how actually such a password authentication scenario works:


The entered password is first encrypted at the client side using the Public Key ('public key1' in the above diagram) and then the encrypted password reaches the App Server where it's decrypted a corresponding Private Key ('private key1' in the above diagram). App Server also fetches the password stored in the database, which might need to be decrypted using another Private Key ('private key2' in the above diagram). Now, the implementation of the algorithms and the generation of the keys should be such that both the decrypted passwords 'decryptedpwd1' and 'decryptedpwd2' should match equal for all the valid cases and they should be unequal otherwise.

How can the encryption be implemented? Will JavaScript/VBScript suffice?

Next question arises, how can we do it effectively and at the same time without consuming much time? The fastest possible way would probably be to have some mechanism in place so that the encryption can take place at the client side only. Okay, so how can the encryption take place at the client side? If we put both the encryption also definition and the public key in the JavaScript (or VBScript) code then one can easily see everything just by viewing the page source. Did you think that making the JavaScript external can solve your problem? As in you only declare the JS file in that case and not list down the contents. Well... if you did think this, you got to think again. A external JS file is equally exposed for viewing by the clients as you can simply type the path (if the path is absolute OR just append the relative path to the roor URL) in the browser window and the complete JS will be there for you to be viewed. We'll see examples below. Evidently, the encryption won't really carry any benefit here.

How else can we do it at the client side? How good would Applets be?

Yeah... now you got a better way of handling the situation. As we all know that applets are also downloaded to the client machine and are executed at client machine itself, so we can now make use of the Java programming language and its security mechanism for having the encryption implemented in a far better manner. What's probably an even more appealing about this approach is that you can NOT see the source of the applet class(es) directly. They are normally shipped in JAR bundles which gives an extra layer of security. You can claim that since the JARs are downloaded and the .class file of the applet class runs within the vicinity of the client JVM so the bytecodes would certainly be available which can then be de-compiled to have a look at the source. Yeah, you're are right that the bytecodes are available at the client machine and it can be decompiled. But, what makes this approach better than the JavaScript approach can be understood by following points:

• Bytecode tempering is automatically detected: if an intruder somehow gets hold of the bytecodes and changes that, the changed bytecode will throw an exception while running whereas any such changes in the JavaScript (or VBSCript) source won't be detected.
• Security Mechanism of Java is much more versatile: what we talked about in the above point is also an integral part of the Java Security mechanism, but it's not only limited to this. It's quite versatile and layered. No such benefit is available with JavaScript (or VBScript).
• JVM offers a far more consistent environment: bytecodes run within a Java Virtual Machine which obviously offers a far more consistent and stable environment as compared to the environment in which JavaScript (or VBSCript) code runs.
• Impl of different public/private keys for every new request: you would probably be aware of the Secure Key concept which forms a part of the password on many systems. The underlying idea in such implementations is to have part of the password which keeps on changing on a continuous basis and thus making it virtually impossible for the attackers to guess that. Similarly, if we want to step up the encryption strength to an even higher level, we can put in place different public/private key combinations for every new request.

Now that we have understood the underlying concept of password encryption, let's move on and see a pictorial representation of how password encryption has been implemented in a real world live scenario. We'll take example of SBI Net Banking and try to understand how the user entered password is getting encrypted there - Diagrammatic representation of Password Encryption >>

Liked the article? You may like to Subscribe to this blog for regular updates. You may also like to follow the blog to manage the bookmark easily and to tell the world that you enjoy GeekExplains. You can find the 'Followers' widget in the rightmost sidebar.

Pictorial rep of Client-side password encryption

This article is a part #2 of the article 'How does Password Encryption work in real world?'. If you have landed directly on this article then you would probably like to go through the first part of the article - Complete working of Password Encryption >>

Diagrammatic rep of how password encryption works for SBI Net Banking

#1: typing the Login URL of Online SBI will get you a web page which will have a JS function named encrypt() and an applet named encryptApplet. Find below the code-snippet as obtained from the Page Source of the Login page:


#2: Once you enter the password and click 'Login' button, the entered password first goes through the basic checks (minimum and maximum length) and if it passes that then it is encrypted by the applet before it's sent to the web/app server. Notice that the public key id is set as it travels to the server as a hidden key which is where used for identifying the corresponding private key id for decrypting the password. This makes the web app implement a different public/private key combo for every new request. Find below the relevant code-snippets doing these tasks:



#3: see the change in password length before and after pressing 'Login' button which actually shows that the encryption is taking place before the request being sent to the server. Notice that the password is encrypted when the 'Login' button is clicked (it turns grey when clicked). Clicking the button first performs the basic validations, then the password encryption, and finally it submits the request to the web/app server.


#4: see below a snapshot showing how an external JavaScript code looks like when opened in the browser versus how an applet JAR file opened in browser looks like. Evidently JavaScript code is easily visible as it's plain text. Whereas, opening up the downloaded JAR/Bytecodes will mostly have special characters and you got to try hard to get hold of the source, if at all that's possible:


Note: On the face of it (by going through the HTML source of the Login page), this is how the password encryption process seems to work for SBI, but this is as per my understanding and of course I can't claim about the actual process. Anyway, the intention here was just to discuss a typical Client-side encryption process.

Liked the article? You may like to Subscribe to this blog for regular updates. You may also like to follow the blog to manage the bookmark easily and to tell the world that you enjoy GeekExplains. You can find the 'Followers' widget in the rightmost sidebar.