(This site links to Amazon or other affiliate links - Info Here )

Sunday, October 27, 2019

What it was like to black box reverse engineer Windows on the WABI (windows on unix) project at Unix System Labs.

What it was like to black box reverse engineer Windows on the WABI (windows on unix) project at Unix System Labs.  A copy of the manual is here.

This is my best recollection of of the WABI project.. since it was over 20+ years ago it’s as close as I could recall.

Unix System labs was a great group of engineers passionate about software engineering and definitely inspired me as a developer in the early part of my career. Even though Unix at the time didn’t run on all hardware if you brought in your own pc in and tried to get Unix running on it everyone was super helpful.  There was an email group you could send to with install issues and someone would come running down to your office if you emailed it to help you troubleshoot an issue.  The offices were shared two person offices which was way better than my previous consulting gig that had been cubes or just flat desks sort of open office style. 
Our team was part of the desktop group and we were working as part of a consortium of companies to work on the WABI project.   Sun Microsystems seemed to be the leader developing the core functionality to take the windows binaries and actually use them to run on Unix.  Our team was to develop the windows 3.1 apps you would normally get with a basic install. Things like the Write application, Program Manager, common dialogs dll, and later the control panel. WABI wasn’t supposed to stand for anything at all but internally it was known to sort of mean Windows Application Binary Interface. 

The computer setup we had was interesting, we had 2 computers one windows computer to build our applications on and to use to see how windows really works. (Plus I think this was a common setup for email which I think used something like Groupwise or something like that on windows)  We also had a unix computer to work with for testing.  We had a small team lab of PC computers maybe 10-20.  There was a larger group shared lab that had 2 to 3 times as many as our teams lab.  There was also a language lab that had a machine running different languages.  We used Xwindows  to be able work remotely from the machine even back during this time.  Which was rather new to someone that just worked on windows at the time.   (Of course could use terminal or telnet for remote).  For windows machines we used VNC at the time, not sure if I realized that was invented by AT&T at the time I believe.

One of that apps we worked on was the program manager application (We called ours application manager) it had been mostly implemented by another engineer but needed refinement to match the windows version.  Some of it was getting Dynamic Data Exchange (DDE) to work properly for installation of new apps and the right files/folders to appear.  That took a large amount of testing of installation of apps.  We were targeting the top 50 applications or more for windows at the time to run.
Our way of reverse engineering was only allowed to use publicly accessible information.  No decompiling.  So we had to use public API docs, published books, or visible/api responses to inputs/outputs.  We would just write test apps to call any api’s and try them and see what they returned.  If a commercial app failed to work we would debug and figure out why and test the api’s to see if they worked differently than the public info.  We found lots of behaviors that didn’t match the documented specs.  We got to speak to some authors of windows internals books to clarify what they wrote which was kind of cool.  We would then mimic the actual api behavior or interface as best we could.   One example was program manager had a feature you could use the keyboard to move the cursor on apps within folders even when they weren’t organized by rows and columns hitting the arrow keys would jump to seemingly the closest file to the left (or in whatever direction the arrow was pressed.)  So me and another engineer worked on an algorithm to mimic that behavior as close as we could.  We compared visually the behavior under various arrangements of files and got an algorithm to operate as close as we could to the program manager behavior.
One day we got a call to help look at the control panel app that was being worked on by Sun Microsystems.  They had generated the app using some sort of UI to code tool and the code was so obtuse to be nearly unmaintainable.  It was a monolith with all functionality built in.    We re-implemented it and simulated the public protocol to support control panel applets (plugin dlls) to make the shell not much more than just loading the dlls (cpl files) and break up the functionality into the dll applets. Sun needed it done quick for some reason and we threw most of the framework together for it over the weekend.
We spent long amounts of time testing installs, the common dialogs, the write application so everything worked just like it did in windows 3.1.  Then we would spend weeks code reviewing with the team so we could be ISO 9000 compliant.  We did written formal code walkthroughs.   We would print out the code and hand out packets to all the reviewers sometimes days before hand to allow preparation and review to comment on the paper.  Then the original coder would walk the reviewers through the code and the reviewers would point out issues or ask questions.  We were not allowed to exceed 500 non commented lines in about 2 ½ hours time period (which wasn’t a problem usually since the reviews were quite involved). There were times we spent almost 2 months just doing code reviews.
We spent many, many months getting everything ready to work and then we heard Sun gave in and said we would be required to install windows and users to have a license to windows in order to use WABI.  Some agreement with Microsoft we presumed to prevent a lawsuit.   So effectively almost all of the apps our team had worked on would be overwritten with the real things on a windows install to the unix system (except I think for control panel since that had to control for some specific unix functionalities too.) 

There was kind of a funny story one of the managers had gone on a plane trip I think to California – on the plane he decided to change seats since his seat was all the way in the back.  Later he heard that if he hadn’t changed his seat he would have been sitting right next to Bill Gates.  We always wondered what kind of conversations might have taken place if he hadn’t changed his seat.
There were folks at Unix System labs we were working with that had worked with Kernigan and Richie. As a software engineer that was pretty cool to hear the various engineering stories about things that made computer history.
Unix System Labs was bought by Novell.  The WABI project started to die down a bit.  We were asked to work on NetWare. 
At some point we got asked what about building a 32bit Unix on windows functionality for some secret project Ray Norda was funding so some of our team went out to Palo Alto to meet with them.  We had started building some of the 32bit windows core functionality on our team as a proof of concept and met up to discuss plans.
We didn’t end up working with the team in California much after that though.
Eventually Novell laid off all the former Unix system lab employees at our whole site but got 3 companies to pick up the employees they wanted BEA Systems, SCO and Hewlett Packard.  We all worked in the same building for a while.  Hewlett Packard was so nervous about other companies in the same building/floors they set up guards at hallways and motion alarms on the hallways you had to tap an electronic fob and run down the hallway to not set it off. It was definitely an experience.

Working on Unix with folks was a great engineering experience and I’m glad I got the opportunity with such a great group of engineers it definitely shaped my engineering career going forward to see how an engineering group is run.  Given they also sold source code for Unix every line of code was scrutinized in code reviews to make sure it was up to par with the quality they wanted to produce.

Also one of the things I took away from that team was not only the passion for software but the passion to promote your fellow team members work.  It was common for team members to share interest in others work and promote others work to other teams/managers, etc.  It might just have been something like “Hey did you see what Joe was working on?  He got a new thread management system to work that saves 20% on cpu cycles.”  Everyone was all about helping each other succeed.  So not only was it a great product and project to work on but a great team and environment as well.

No comments:

Post a Comment