Humbled

If you have ever had a missions experience then you most likely know the feeling of humility that comes from being treated with hospitality from people who have little and whom you are there to serve.

You go with the idea to give and yet you are given so much. That has been my experience here in the Solomons this week.

I'm honored to have been invited here, sent here,housed and fed here. I'm aware if the rare treat to be in a place with such significance during WWII.

Working hard is the best way I know how to thank God and thank the folks here for the many blessings that come with thus opportunity to serve.

This is the market here. These are the people seeking food. These are the people we are trying to reach with the bread of life.

Runnin cable in Solomons

Found this

Part of what I am replacing is the outside waps. Found one in the rafters on a deck with an external antenna connector. But to get power to it they did some splicing and I assume rats or geckos liked the tape glue because it was off and the two power leads were touching. I don't know how the wap wasn't shorting out. But it may have played a huge hand in some of the network faults. This has been replaced with nicer wiring and a wap powerful enough to keep indoors. The wap had geckos all over it staying warm. I would have snapped a pic but it was too dark out.

What I'm here to replace

Honiara harbor

Choices choices

Pop cone!!!

Family Wagon

This is a 'tuktuk' popular in India. It's a 3 wheeled vehicle, powered by a small Diahatsu truck engine. It's a manual transmission. Left foot clutch, RIGHT foot brake, right hand is the twist throttle, and the stick shift rests between your knees.

My wife and kids love this vehicle. My wife, who 4 years ago didn't know how to drive a stick shift, now can drive a tuktuk with ease!

We had this unique family vehicle for a short amount of time and then passed it on to a family who needed it more. For me, driving it was a pain, literally. I had to bend far over to see out the windshield and crouch down. Which is why we passed it on to someone who would use it more. Still, it was a nice way to stay dry in the rain for getting around centre.



THIS is why I didn't enjoy it. They call me Mr. Incredible when I drive it.

Mr. Incredible

Family Wagon

the Family Wagon

Solomons Islands

Honiara here I come.

Our organization has a branch in the Solomon Islands. Currently they have no I.T. guy and are in need of a network across their center to improve communication. Good communication furthers the work of Bible Translation in many ways.

at the business level:
-internet access speeds access to information and resources
-communication to remote areas can put people together who to get the job done quicker
-backing up of data avoids having to re-do work when laptops are damaged by weather, or lost to theft or accident.

on a personal level:
-you feel less remote when you know you can email or call a loved one, a prayer partner, a financial partner.
-overall sense of connection with the different worlds we move between helps decrease the sense of isolation, which in turn, increases the ability to stay in a remote place longer and get more work done (good communication helps to DECREASE the 'island fever' feeling of being all alone and wanting to go home)


Those are simply some highlights of why communication is so important. Some people use it rather creatively and can do multiple translation projects from anywhere in the world!

I'm pretty sure since you're a blog reader, I don't have to fully sell the idea of a reliable connection to the world. Even if it is only 256k download speeds.

Well, as a favor to this sister branch, our directors have lent me to them to go and:
-install wireless networking
-install network printing
-install file server/backup solution
-install internet routing/firewall/billing system (this is my own custom home grown solution that we're using, and so far, it's been very useful. My manager was asked to give a presentation at a local tech conference on it. Fully opensource, best-of-brand (so to speak) but also customized so that administrators can easily create accounts for people to use their internet with billing. (think of it as a low cost Metered Hotspot).

It's rather complex actually, but cool.
I run VMware ESXi on top of a small PC. That serves 3 virtual machines. Two of which are PFSENSE (opensource firewall, opensource captive portal) and 1 is Ubuntu 10.04 running FreeRadius and a customized version of Daloradius). It can plug into an LDAP domain or Active Directory domain, or, simply use it's own local Mysql database of users.

It's really something I'm proud of having been able to piece together and document. To get the effectiveness of this 1 box, you'd have to pay at least $2000 USD for something akin to what Starbucks uses to provide you with wireless. My cost, between $400 and $630 USD!!

Anyway, geek moment aside...

I'm heading to the Solomon Islands on Monday!!! How cool is that! The sister branch is paying for me to come out there, work my tail off for 1 week, and then come back.

They have no I.T. person, and could really use one, but my hopes are that when I'm done, I'll be able to remotely support them.

I'm telling you, it'll be an adventure for sure. Flying internationally always is. I'll be leaving behind my family for about 10 days. Please pray for them.

Please pray that the trip goes well and things don't break, and that when I'm gone it keeps working well for them. That's the trick. Designing and implementing something that will work for a very long time when I'm away.

Well, time to go pack.
Can you imagine... it's like I'm becoming a world traveler or something! ME!?

I'll definitely take pictures!
I think it's too much to hope for any actual site seeing time, there is too much work to do in too short of a time. But maybe I'll get some good pictures of me setting up the network! That's what you want to see isn't it? No one really wants to see tropical landscapes, and such. You want to see a sweaty guy sitting in a dark room, splicing cables!

Prayer and Hardware

Mock me if you must, but since high school I have believed in the power of prayer on hardware. Since that very first time I was troubleshooting a broken professional tv Video Tape Recorder. The teacher said 'Chad this isn't working see what you can do.' I spent an hour on it, couldn't resolve it, put my hands on it and prayed. It started working.

We don't battle against flesh and blood alone, but in my job it seems that is my primary function. To battle against the physical and logical failures of computers and networks.

Today we had the most catastrophic failure here that I've ever seen.

-one of our VM servers had a hardware error which crashed 8 virtual machines

-when the second VM server came up, and brought them back up, they were in a confused state because the boot sequence was broken (our Active directory came up last)

-at the same time, a motherboard lost a sensor and refused to boot on our main backup server... which means, not only would we not get a full backup tonight, but backups last night probably failed.

-all of this, on top of massive email failures, because one of our smarthosts went down and was refusing connections and I had 900 email messages in the queue.



In short, almost everything was broken at the same time, and I could not find a common cause. Typically you could look at power, but that wouldn't explain why the datastore filled up.

So as we sat there, working, trying to get it back online, I got to a point where I said 'wow! This is a lot to fix.' The phones were ringing, I was both network manager and acting department manager.

I was stuck. I had to fix these things with my co-worker, and had to do it fast.

I stopped and looked, and closed my eyes and prayed.
I put out a quick shout to some of the people calling in to tell me what I already knew (that it was broken)... and said 'all you can do is be patient, and pray'.

"God, if this is some random act of failure, please give me the ability to solve it quickly. If this is a spiritual thing, I pray against the enemy and his attempts. You are greater than he, and I pray you would bring healing to this equipment, bring it online, and I thank you for the skill you've taught me over the years, please make my brain work well right now."

It wasn't even 2 minutes before things starting coming online again.
I can't explain to you the joy that comes from seeing all reds, to all greens, in a very short amount of time.

In the end, we still haven't been able to diagnose a single root cause. It seems like multiple simultaneous failures.

We lost 1 server, the motherboard is dead.
But, as God would have it, we had another server recently offlined (we virtualized it), so we could steal it's motherboard.

We brought the crashed backup server back online in 2 minutes or so, and guess what.

The crashes happened in such a way last night, that we got all of our backups!

Not only did stuff come back online, but when it went OFFLINE, it went in such a way that we still did not lose a single bit of translation data, and all the other data that enables this work. And email is routing properly now too!

Some may cry 'coincidence'. Others may say 'well that's why you have I.T. guys'.

I say, God is great. He spared us.

It could have been really really bad, and I think, it would have been if we didn't realize we needed to pray. If we didn't submit and try to continue on in our own way, sure it may have gotten fixed eventually but God's way is so much better.

It is SO awesome that we're back online, what a crazy morning, but it ended without sorrow and that's a good thing.

Death of Music

This is my M-audio fast track. I bought it close to ten years ago in the U.S. and use it to interface a microphone and a guitar pickup into my mac. I use garageband from there to record tracks and goof around with music. My daughter and son LOVE playing with garageband, and I haven't done vocals for about 1.5 years. So my little fast-track sat in a locked cupboard, next to my microphone.

The other day my daughter and I had an idea for a fun project and I got all the equipment out and set it up for her to sing into, and something wasn't working. After trying a few things, I opened up the little box for the first time and saw this

You may be confused as to what you're looking at. Look for the little piles of dirt. At first I thought it was cockroach poo, but then upon conferring with my son, it's dirt, and dead ants.

Corrosion in electronics is common, notice the copper plate at the top there? Copper corrodes quickly here. Also if you look at the bottom pic you can see light green corrosion near the bottom. This type of thing happens from time to time here. Electronics that are stored, can suddenly become some pest's home. Others can be subject to corrosion from the humidity. The electronics you leave on, can become nice warm places for interesting things like geckos to lay eggs in.

We have what we call a 'dry box' for important electronics. It's a wooden cupboard with a light bulb in it and a fan. That is usually enough to keep this type of corrosion at bay. Although we do find it much more common near the beach (more humidity and salt in the air).

I took several Q-tips and some rubbing alcohol and clean out all the dirt but the corrosion was unrepairable. So I'll be replacing the unit somehow. Still it lasted me several good years, and I can be happy with it.

I have a backup plan, there are some electronics gurus here who could probably fix it, but the time involved would be quite a bit. I think that it might have been salvageable if I had caught the corrosion before plugging it in. I think giving it power zapped something inside that no amount of cleaning will bring back to life.

And this, is only 1 more reason why having technical people here to support the work is important. Someone needs to be able to open up your laptop and point to the big pile of dead ants and dirt and say 'Well, THERE's yer problem!' ... and then fix it.

Tim Tam

...from life in PNG

came across this pic from fail blog and thought it pretty funny.

Arnotts is an Australian company that makes 'biscuits' (Aussie for cookie which is funny if you ever invite an Aussie over for biscuits and gravy, you'll get an odd look 'why would you put gravy on cookies?')

TimTam is a chocolatey flakey cookie that the kids love. Some will actually drink MILO (chocolate flavored energy drink) through a TimTam. (they also eat ramen noodles dry).

So this picture had me laughing. We don't often buy these, because being imported from Australia, they're a little pricey.

Jury Rig of the Week

This is a close up of a FIXED Australian cistern.

A few days ago, our septic overflowed. Septic here is often a small tank, that leeches into the ground. Our septic was full because our ground is so saturated with water from the rains, that it doesn't leech very quickly. The water table is very near the surface, and as such, we depend mostly on the bacteria in the tank to eat the septic up and leech the water.

(for those unfamiliar with how a septic system works, don't worry I'll spare you further details (-;).

I tracked down the cause of this overflow by two methods:
1 - I had no idea where the tank was since I never installed it and found out by following my nose
and
2 - the lid wasn't below the surface, it was near my bedroom window.

I kept telling my wife as we go to sleep at night for a week, 'that funny smell, is NOT me... what is that?'

Long story short;

The toilet had broken, and was flooding water very slowly into the septic tank, filling it up. A small plastic piece, specially shaped to stop the flow of water when the float bob raised, had broken in two.

EASY fix right? head on down to the local hardware store and buy the internals to a toilet.

WAIT.. no one sells the internals to these things?

Okay buy the top half of a toilet then.
$75.00 and a 6 hour drive. Okay crazy glue? Except the crazy glue here doesn't work like it's supposed to, it comes out tacky instead of liquid possibly because of the heat, and once you open the tube, it's worthless the next day. So I don't keep it around too much. I considered duct tape, but tape and water don't mix well.

I asked around if anyone had half of a broken toilet I could scrounge from, but I couldn't find the right piece from those.

A 25 cent plastic piece broke. THERE HAS to be a better way.

So, I scrounged up a thick bit of old sheet metal, looks like it used to be used for some sort of radio bracket. Traced out what I thought the piece should look like.

Took an angle grinder to it, and made a small 2 inch piece of metal, specially shaped with a hole through it... mounted it in place of the cheap plastic piece that broke.

and.. viola... FIXED TOILET.

I was pretty proud of my fix. Admittedly, first I tried wood. I always try a wood fix first, but then I quickly went to metal, seeing as how wood rots slower than metal rusts.

You can see the piece in the pic above, top right. It's shaped like a little tiny noched pistol, but you can't see that part.

That's my jury-rig of the week.

I'm proud of myself for having been able to shape the piece and cut it precisely enough to fit where it needed to go.

Total cost $0. saved myself $75.00 and a 6 hour drive!

I came bursting out of the bathroom, beating my chest and raising my arms in triumph.

"Wife of mine, children, gather around, I have accomplished a great feat!"

and my wife replied,

"that's great dear, but most of us conquered that activity when we were five."

pa dum ta.

no she didn't say that, but it would have been really funny had she.

what she actually said was
"okay?!?!?"
"I fixed the toilet!"
"it was broken?"

doh!
Well, jury riggers unite, revel in my newly fixed toilet, and stay tuned for the next jury rig saga.

Fixer upper

This is the cistern (top part) of an Australian toilet with the lid off so you can see the inside. These toilets have 2 buttons. 1/2 flush and full flush and while standard here, that type of toilet is making its presence known in the U.S.

Aitape Online

As part of our goal to bring our regional areas on line, Kurt and Wayne (co workers) fly to Aitape and successfully installed a VSAT internet connect.

256mb down
512mb up
(blazing fast I know!)


Later this month, I'll be heading to Buka to do the same thing.

(from left to right, Kurt - fellow network engineer, Ben - translator and his co-translator whose name I can't recall, and Wayne - radio/electronics engineer)

Fridays

Fridays around here are really super busy and exciting and I never have figured out why.
But this week I'm acting department manager, which means I'm the guy in charge of the craziness.

RAIN RAIN RAIN
KABOOM, what was that?
I dunno I can't find out right now I have to get a guy out to Aviation to fix their printing, and return the autoshop's rental truck.

On the way, I find the industrial manager looking up, so I stop.

AHA!!! the the kaboom was lightning having hit a tree, broken a branch off, the branch took out a power line.

SEE the light green on the branch? It is most likely the path the lightning took down it,taking a strip of bark with it, but it could also be the branch sliding across the power line as it dangled and took it out.

HARD to tell, but it definitely fell, and it definitely took out the power.

People are running around trying to keep their homes from flooding.

It's a CRAZY day.
WOW.

It is at times like these, that I remember the adrenaline rush of working in I.T. in a third world country. I.T. can be exciting in Corp. America, here, you're working without a net, and soaking wet.

Kaboom

Beautiful sunny morning.
At lunch, it turned into a torrential downpour. Heavier than usual.
I headed back to work, got soaked in the process, to the bone.
the roads were rivers, my yard was a lake. I saw no grass, and I contemplated the chances my home would flood.

Lightening, thunder... and was about to post to the community to unplug their electronics to avoid lightening damage when

CRACK!!!!
lighting and thunder at the same time, had to be less than a thousand feet away from us, probably hit a power pole, because our radio repair room shot sparks into the room. (I was outside, didn't see it)

Anyway, definitely expecting some repairs due to lightning damage.
VERY thankful our servers are on DC power.