Diary of a Network Geek

Well, it seems the “sure thing” contract I thought I had lined up is far from it. I’ve been chasing after a contract upgrading a local school district for almost four months now. A little over a week ago, I was told that schools “in the area” were getting their money. Apparently, the person I was talking to thought that New Mexico and Houston, Texas are in the same area, because the government money was flowing there, not here. Behold, the joys of nation-wide recruiting firms!

So, in the meantime, I had told a different recruiting firm that I wouldn’t shave my beautiful beard off for a contract-to-perm position that paid roughly $13K less than my last job. Guess what I did yesterday morning? Yeah, that’s right, I called back telling them I’d changed my mind and if the nice, beard-shaving people with money were still interested, I’d talk to them. Frankly, I think it’s too late, which is a shame because the position might have beefed up my Windows 2000 experience. Not to mention my Exchange exposure. And, I would have still been using Novell, too. Sure, they’re phasing that out in favor of Windows, but there’s a lot more work in the Microsoft arena these days anyway. Shoot, there might even have been the possibility of getting my MCSE! But, despite swallowing what little pride I had left and just about begging for an interview, I don’t think that one’s going to work out.

Hey, buddy, can you spare a dime? Or a job? I’d settle for a job.

Well, it sure has been a long time since I did any updates on this page! In fact, it’s been literally months!! I can explain…

First off, I’ve been pretty damn busy at work. After my little adventure in the New Orleans airport, I hardly had a chance to sit down. Not only have I had to roll out two sets of patches, or “Service Packs” as they’re called now, on our Novell servers, but I’ve had to set up a Novell BorderManager Proxy server. Man, the troubles I’ve had with that! The initial setup was okay, but getting the VPN (Virtual Private Network) working has really bedeviled me. It’s been so bad that my boss, the CIO, had to raise some ruckus with our sales reps at Novell. He’s finally gotten me a connection with our local Support Engineer, who’s promised to come take a look at our configuration for me. Hopefully, he can find what a CNE (Certified Novell Engineer) with more than eight years experience has missed.
Second of all, I finally got my gorgeous girlfriend/fiancé to marry me. On Saturday, March 10th, 2001, Anne and I got married in the Canterbury Wedding Chapel at the Excalibur Hotel in Las Vegas, Nevada. Allie, Anne’s daughter, was our ring-bearer/flower-girl. The service was nice, short and attended by the photographer. We thought it was the perfect way to get married. No one hassling us or telling us what to do and how to do it. No meddling mothers or aunts or sisters to make us miserable. No seating charts or receptions to worry about planning. Just a cool, quick wedding and a great steak dinner afterward. After that, we spent most of the week running around Las Vegas doing tourist stuff. We went to see Sigfried & Roy, Tournament of Kings and The Blue Man Group. We also went to see Star Trek:The Experience, where I bought “genuine” Romulan Ale and Klingon Blood Wine. But, of course, work intervened and I had to come back a day early to “fix” something that had gone wrong. What would they do if I ever quit?
Finally, now that we’ve done all that, we’ve been looking for houses. It looks like we found a nice four-bedroom, two-bath, two-story house on a lot that’s almost double-deep! And, even with all the flooding that we’ve had here in Houston, it stayed dry. What could be better?!? Two koi-ponds, that’s what. That’s right, the property already has two koi ponds on it, complete with koi. There’s a fireplace with a Ben Franklin stove in it and a partially finished garage. There’s a covered car port that’s connected to the house by a covered walkway. It’s really great and I hope the sale goes through okay. We’re scheduled to close this Friday, 6/29/2001!
And, of course, I’d be remiss if I didn’t mention my kidney stone. That’s right, as if all the work I’ve been doing isn’t enough, I was doubled over in pain by a kidney stone last week Monday, 6/22/2001. It got so bad, that I had Anne take me to the emergency room. We ran into Charles, our paramedic friend, there who told the nurses to take good care of me, which they did. I got moved through as fast as I’ve ever been. And a nice, young doctor gave me some great pain medication. I still haven’t passed the damn thing, but I’m off to see the friendly urologist tomorrow. I plan on demanding that he “do something” about it because I’m tired of peeing through a strainer!
Until next time….

So, you’d think with all that this past week, things would be getting better, right? Well, think again. Apparently, the network has more issues than even *I* was aware. NDS (Netware Directory Services) and GroupWise (the e-mail program) were way out of synch. The bad part about this is that someone, who works for me, kept resubmitting an update. In short, we tried too many updates without out allowing these two systems to synch up and now they’re broken. Mind, it didn’t help much that we didn’t find it until things were really bad. I think that was due to my being absent from the office. I feel like I’m bridging a gap of some kind in the department communications. There’s a guy there who’s very knowledgeable about networks, if not NDS in particular, but has trouble communicating with his co-workers. To give you an idea of what this means, someone congratulated me for being able to work with him the other day.
Anyway, that means that this problem was probably known, but no one did anything about it, because they were afraid to try and work with this guy. The sad thing is, he probably could have fixed this problem if he’d known about it soon enough. Ah, well, what’s done is done. The damaged GroupWise database has been sent off to Novell, and we should hear something back on Monday. It probably means that I’ll be working late all week again. I’m not sure how it is that this always happens, but we’re going to see if we can distribute the “fun” around the department. I’ll let you know how it goes.

Here I sit, at 12:25am, in the lobby of the New Orleans airport waiting until 6:00am so I can go home. “Why?”, you ask. Well, I’ll tell you. If for no other reason than to look busy so that the nice police officers stop looking at me like I’m some kind of vagrant. It all started a couple of weeks ago, when I noticed that the chuckle-head who was the Network Admin before me had not gotten all the servers onto the latest set of service packs. When I asked my boss about this, he was somewhat surprised, since he’d been told that everything was up to date. Not hardly. In fact, I had servers that went back at least two revisions! So, two weeks ago I prepared to get all the servers updated. Behold, my true troubles begin! I try to get the patches copied to all the servers and discover that it takes literally *days* to copy the files! That’s right, the patches are so big, and our bandwidth is so small, that it takes almost five days to copy the Novell patches to all the servers. But, wait! They didn’t really copy to all the servers!! Yikes! So, the re-copying begins. Now, this is when it starts to get really fun. You see, in most of the cases, the reason the Novell Service Pack didn’t copy, is because there just wasn’t enough room on the main volume. (That’s SYS: for all you Novell CNEs out there. Not that there’s many of us left. Certified Netware Engineers are a dying breed, I’m afraid.) Well, I try the obvious solution and recopy everything to one of the other volumes (that’s a “disk drive” to you non-network engineer geeks.) At first, that seems to work well, but, on closer examination, I discover that one of the servers has still refused to copy everything. It turns out that this server lost it’s wide area network connection. *sigh*
Okay, so I try copying the patches yet again, but this time I send them in a compressed format. So far, so good. While all that copying was going on, I started installing the patches on the other servers. Again, most of them worked just fine, but those few with teeny, tiny SYS: volumes don’t all work. So, at this point, I’ve ended up with four that don’t quite have enough room on the main volume to install patches and one that finally has the patches *copied* , but not installed. This is where the problems that actually led me to the New Orleans airport began. We recently purchased a nifty utility called ServerMagic from PowerQuest. This darn program is the best thing since sliced bread! Among other things, it lets you resize Netware volumes without having to destroy them first! Totally cool, and totally impossible before this tool. So, never having used this before, I installed it on one of the remote servers. I ran it and it asks to reboot the server. (GULP!) I took a deep breath and tell the program, “okay”. Bang! The server goes away. Poof! It never comes back up. *sigh* Well, that was just the beginning. I let my boss know what was going on, and then we sat back to wait for Monday morning when that office discovered their crashed server. In the meantime, of course, I installed the patches on the rest of the servers. Or, at least, I *tried*. Some of these servers **still** didn’t have enough room to install all the patches!! Well, I figured that I’d eventually get the space problem worked out, and just left it alone. Oh, boy, was that a mistake! Monday, the first office called to get the server up and running. We got the first signs of life about 9:30am, with some errors, of course. About then, that office’s answering service lets them know that I know about the server and that they should call me. *sigh* So, okay, we get this pretty well squared away, when the development department tells me about this new, totally redesigned piece of misson-critical software that *I* need to get rolled out. Being the kind of guy I am, I start getting this all configured to run, from scratch. Now, we’ve survived until Tuesday morning without a tragedy.
Well, about the time that I start to work on finishing the big, new, shiny software rollout, another site calls to tell me that their server is down. “How did that happen?”, I wonder. No time to worry about that, though, because we’ve got two whole companies down while this one fileserver is off-line. So, like I always do, I started trying to figure out what happened so that I can undo it. It turns out that they took it upon themselves to reboot the server because there were users having problems logging in. Aha! The new patches had partially loaded, because they were partially installed, and some of the new files didn’t like some of the old files. Blamo! One crashed fileserver. Well, I banged away at it for most of the afternoon, one way or another. I tried loading files from backup directories, and copying files from the backup directories, neither of which worked. Then I tried to use ServerMagic to resize volumes so that I could come up with the extra space I needed to install the patches. Well, that *almost* worked. Apparently, the old disk drive just couldn’t quite handle the new utility and just decided to stop working.
By this time, it’s after 4:30pm and I’ve already figured out that I’m flying out to the remote site. Shortly after I suppose I’m going, my boss confirms it for me. We agree to try a couple more things, but not past 5:30pm. So, now, it’s do-or-die time. Let’s just say that I didn’t “do”. My boss helps me get the travel arranged, and we decide that a day trip should be enough. I gather my tools together and scare up a 4Gig external drive that I can add on to the ailing server. Then, I got the brilliant idea to burn a CD-ROM that has the patches on it. After all, since I have to add diskspace anyway, I might as well install the patches while I’m there! That was easier said than done. *sigh* I finally downloaded the required files to my PC and used my own CDR to make what I needed. It took over 6 hours to get the complete download. I got out of the house with just enough time to make the first flight from IAH to OLY. That’s at 6:50am, just in case you’re interested. It gets into New Orleans at about 7:00am. I then spent almost 45 minutes getting from the airport into my rental car. After that, I was ready to drive more than an hour to get to our site in Houma, which I finally did after a few wrong turns.
Finally, I was ready to start working on the server at about 10:30am. It took all of about 15 minutes to get the extra drive installed, configured and running. The fileserver itself took a little longer. I got enough of the new drive allocated to the SYS: volume and then I got the new patches installed from my CD. So far, so good. In fact, at this point was planning to try and catch an earlier flight home than the 5:20pm flight I was scheduled to fly back on. Oh, well, maybe next time. I rebooted the server, so that the new patches were activated, but started getting errors right away. Apparently, the USR volume, where most of the actual data existed, was damaged. Damn! So, I try every repair utility that I can think of, without any improvement. Eventually, we had to delete most of the data because it was too badly damaged to actually use. Unfortunately, that volume also stored the GroupWise e-mail system data. Whoops! So, now, it was about 2:00pm and I was wolfing down an oyster poboy while trying to get the USR volume back online, when it hit me to try and use ServerMagic to fix the problem. Well, that *almost* worked, but it recovered the drive at the expense of the data it contained. Double damn! We don’t back up the GroupWise data for legal reasons, so now we can’t restore it. But, wait! The local remote admin ran a backup last month that accidentally included the e-mail directories. Hooray!! Well, to shorten a *very* long story, we managed to recover and rebuild the mail databases so at least they’ll have user accounts and mail going back a couple of weeks. Not ideal, but better than nothing.
But, how did I end up missing my flight? Well, to prove that we give the best customer service, I stayed while the Arcserve restores ran. (Okay, it was more like they ran, then got totally screwed up, then we deleted them and recreated the jobs and *then* they ran.) And, the next thing you know, it’s 9:30pm and I’m running to try and make a 10:30pm flight. Obviously, I missed it. *sigh* On a more positive note, I do have a flight booked for 6:00am. Of course, that means that I’ll have just enough time to shower and change before I head back into work so that I can get the super-duper software rollout working. Blech! I plan to get the most out of the damn company picnic on Sunday!

Well, last week, I saw black helicopters flying over fields West of my office. Now, Anne thinks that I’ve been watch too much First Wave, but I swear there were military helicopters flying around out there. Gosh, it’s weird having a window… I saw all that, incidentally, because I was using my second machine to get the screen captures that Novell wants for my corporate licensing agreement self-audit. (Oh, boy, is my job fun, or what?) Anyway, that machine faces the windows in my office. Well, for those of you that care, I’ve just about gotten my new bookstore done. I decided to redo it from stem to stern, as it were. Not only am I doing away with the JavaScript menus, to make it easier to navigate without having any security concerns, but I’m adding a lot of new books! Don’t worry, though, all my old favorites will be there, too. On another note, Allie, Anne’s daughter, started piano lessons on Saturday. Apparently, her Nana (grandmother to you Northern carpet-baggers) has done so much work with her that Allie skipped right to the second or third lesson! Good for Allie, but it makes me wonder just what she’s been doing over at Nana’s… Anyway, she seems to have enjoyed the first lesson, at least. We’ll see how long she keeps at it. Hopefully, she’ll stick with it. I’d like to think that music positively influenced my life, even if I don’t make my living via music. Speaking of family, Nancy and Tim, her new beau, are getting married! Yikes! I don’t think her divorce is even done yet!! (Ah, if only her sister would take a cue from that…) So, we’ll be flying up there in October for that wedding. Big fun. Hopefully, Anne will get the time off. I already have, but… Well, the life of a contractor is a little different. I think it’ll be okay, but you just never know. Well, more later in the week, but that’s all for now. Stay tuned for the latest wedding plans and developments.

Okay, so I’ve gotten a little deeper into the network and I’m finding more “little” things wrong. For one, the servers aren’t all on the same version of Directory Services. For those of you who aren’t Certified Netware Engineers, that’s a bad thing. NDS, or Netware Directory Services, is the distributed database of everything, including security, on the network. If all the servers don’t have the same revision of the NDS programs, called NLMs in Netware Land, there can be problems with security and access between servers. Enough of those problems and the network stops running. But, so far, that hasn’t been a real issue. So, I’m not too worried. I’m working on correcting the problem, but it can wait until the next “maintenance weekend” to reboot the servers so this will take effect.
The other “issue” that sticks out for me is a bad time synchronization plan. I don’t think that it got this way on purpose, but time synchronization is all wrong. Mainly, it looks like the last Netware Admin had no experience on Netware 4.x or greater. I won’t go into the details, but… Well, let’s just say it needs work. This is something that I can work on, and implement, without taking the servers down. I’ve started working on that, too. It’ll take a couple of days to get straight, but that will improve network performance quite a bit.
And, of course, there’s virtually no network documentation. Not a big surprise, if you have any experience with taking over a network. How many Network Engineers really want to write documentation? But, I know it needs to be done. And, I also know how good that looks to management. (Yeah, yeah, I know. I’m kissing butt already. *sigh*) Anyway, I need to go get my sleep so I can be early to the office and hard at work. Updates soon!

Well, not much of an update today, but I thought I’d add in some personal stuff. Just for variety.
Last weekend, the girls went out to a Celtic Fest and left me at home. I thought it would be nice for them to get some “girl time” together. Anne has been working really hard and hasn’t been able to see Allie as much as she’d like, so I thought it would be a good opportunity for them to “bond”. This weekend Anne isn’t feeling too well, so I might just end up taking Allie out shopping for a little while. We need to get a book on telling time for her. I’ve instituted a work-for-allowance program that requires some knowledge of telling time. (Okay, so what I’ve done is tell her that I’ll pay her $2.50 per hour of cleaning up and other chores that she does. She might as well get used to the idea young. It is how the world works, after all.) Besides, I also need to get a book on the current version of Red Hat Linux. After looking around the Internet at stuff, it seems like everything for Linux of any real value assumes that you have a version of Red Hat installed. So, in spite of the fact that I’ve been enjoying Storm Linux, I’ll be learning Red Hat.
Speaking of learning… One of the reasons that I’m focusing more on Linux is that my studying for my CCNA (Cisco Certified Network Associate, for the uninitiated) has been slow going indeed. I’m still working on it, but with the lack of sleep that house guests create, it’s been hard to slog through the issues relating to sub-networking. It’s especially bad since I’m already familiar with most of the material! I just have to get the Cisco methods down for the test. Shoot! I’ve been doing TCP/IP since Novell used IP tunneling!! (That’s over 8 years, incidentally.)
Anyway, this is going to be a light weekend. At least, it is if I have anything to say about it. I’m in a good mood since I got two calls from headhunters on Friday. They’re looking to fill Network Manager positions, which is what I think I need to have next. I changed my resume on Monster.Com to reflect the fact that I managed people for the past two years. It seems to have made a difference! So, like I said, I’m in a pretty good mood this morning, so, if Anne is feeling up to it, I may just take my two girls out to our favorite little Greek place for lunch, Niko Niko’s.

See you next week!

Well, now that I’ve finally set up my news and updates page, namely Diary of a Network Geek, I’m going to start adding to it. My dear, sweet girlfriend, Anne, suggested that I chronicle my quest for a new job. I’ve decided that I like that suggestion, so I’m going to start with a little background.
The company I started working for when I moved to Texas, Harbor Financial Mortgage Corporation, filed Chapter 11 bankruptcy in December of 1999. Currently, I’m a contractor for the liquidation company that is disposing of Harbor’s assets. Without going into too much detail about why they filed Chapter 11, let’s just say it was a bit of a surprise. When I joined the company in June of 1998, we were in a period of explosive growth. We gained over 200 employees in less than 8 months, and continued to add employees slowly for a year. Less than two years after I signed on, we’re all but gone. At last count, there were about a dozen employees left here, including myself.
So, you ask, what have I been doing since December? Well, for starters, I separated the Dallas network from the combined Houston-Waco networks. I was in charge of the Houston-Waco end of things. It all happened just like it’s supposed to in the on-line documentation at Novell. Well, after the router guys at Sprint got the right routers disconnected, that is. *Sigh* Nothing’s perfect, I suppose. On the upside, the Houston separation from Waco went without even that much going wrong. Not every day that a Network Engineer gets to purposely break then fix a network!
After that, mostly what I did was maintain the system as it’s been. I kept the backups running and made sure that the sad, little ISDN connection to the Internet was up and running. Of course, from time to time I had to perform some other regular maintenance, but that didn’t take much time. So, in between times, I studied for my CNE (Certified Netware Engineer, for the uninitiated) update test. Which I passed, BTW. So now I’m a genuine CNE 5. In fact, the last time I called Novell, I was the only Jim Hoffman that was a CNE! Cool!
And, the rest of the time, I spend catching up on all the tech reading that I’ve been missing out on. I setup a Linux machine, too, just to brush up my Unix skills, which are far too out of date for my taste. Oh, yeah, I’ve been talking to people about jobs, too. See, I’ll be totally out of work at the end of June. At least, that’s what they’re saying today. Who knows, though, next week they might tell me to get out. Or they might tell me that they’re extending the contract until the end of July! I just don’t know what’s going to happen from day to day.
So, this Diary of a Network Geek will chronicle my trials and tribulations as I go out job hunting. I promise to change the names of the people involved to “protect the innocent”!