November 6 2017
We had a fairly long outage from approx 9:45pm last night into this morning, with intermittent uptime until it finally settled down about 1pm EST. There was a thunderstorm in Columbus with high winds in the neighborhood of our church, which resulted in many downed trees and a brief power outage which was well within the 2hr battery backup power capacity; however, there was an associated internet service outage to neighborhood from approx 7pm which kept the server disconnected from the internet.
The timing was especially bad because about 6 churches have their auctions this coming Saturday the 11th and would really like for it to be reliable this week.
Also, our church administrator was busy dealing with some minor building damage, and did not see my frantic messages until I showed up in person.
So as of now, 1:45pm, all appears to be back up and running and none the worse for wear.
As a result of all this, I will accelerate my plans to investigate hosting the site in the cloud - I think we have enough churches now to handle the associated costs, it's more a matter of me learning exactly how it's done.
My apologies to all for the inconvenience.
Seems like we've had numerous brief outages lately, but hard to establish any real patterns. I've contacted our ISP tech support, who recommended restarting all our network equipment, which we've done.
We did make one discovery which might be relevant to any of you with Joomla based CMS systems for your main website, or any other kind of website where you set up a redirect to link yoursite/auction to www.togetherauction.com/yoursite. It works great, but then someone tried using https to run the main church website, which otherwise works fine. But, togetherauction doesn't have an https certificate, so if they hit that link, it was keeping the same protocol for the redirect, and giving connection refused. So we aren't using those redirects anymore and instead using just simple external links such as you'd use for any other site.
June 15 2016
I've had a number of recent (early 2016) reports of the site not being available. This is odd, because whenever I suggest using a site like http://downforeveryoneorjustme.com/togetherauction.com it always says it is up, but still, there must be some kind of network issue that hides it sometimes.
I've registered a few backup network names to try to help with this problem. If for some reason you can't get thru using http://togetherauction.com, you can try one of these alternates:
I don't suggest advertising or bookmarking any of these, since I may decide not to renew them if the problem is somehow resolved, but they should function identically - I hope this can provide some relief.
You may also want to consider having a plan B to deal with outages at the worst possible moments (perhaps including letting folks go home without settling up or collecting their loot) and doing sale entry when the system comes back up. The act of having a plan will at least make you feel better, and perhaps also help you realize that you would still manage OK...
March 17 2015
Just realized the network upgrade caused all the automatic email reminders to stop working. I fixed that for future events, but all the emails that were skipped are not going to be re-sent. Hosts should
probably not count soley on those emails.
March 9 2015
We decided to switch network providers today from Time Warner to Wide Open West. It's too soon to tell whether this might fix our power outage blues (crossing fingers it might), but anyway the network is about twice as fast, so that should make the images on your catalogs load slightly faster the first time for those of you with fast internet connections.
For several months we've had unusually cold weather and more than our typical share of power outages. We have a big UPS so when the power blinks, the server keeps humming along, but for some reason no web
traffic can go in or out of it. Our theory is that some piece of network equipment 'upstream' is not similarly protected by battery backup.
Sept 9 2011
For several weeks (since approx September 9 2011) we have been experiencing 'intermittent' network connectivity outages. This is a real hassle, and I am working hard to resolve the situation.. but
unfortunately I can't give any concrete information about when to expect things to return to normal. What I do know is that we have replaced several pieces of network equipment at our hosting location in Columbus Ohio, and the problem remains - we are working with our ISP (Time Warner aka “Road Runner” business services) to find the root cause of these connectivity problems. When this is happening, you will be able to still read this help page (because it is hosted externally by Wikispaces.com,
but unable to reach any pages at togetherauction.com. Of course, if this were to continue into your auction 'season' it would be a major concern, so please be assured I am doing my best to avoid that possibility.
Sept 16 2011
We have upgraded all local equipment and problem continues but at a fairly low frequency - our ISP has 'generously' doubled our connection speed in the meantime to compensate. At this rate it is likely
TogetherAuction users should not notice any connection issues. However, if your page fails to 'come back' after a long time, try hitting the refresh button (usually F5).
Sept 27, 2011
We had several outages today, and they lasted longer than usual because I am away on vacation. I had noticed what I thought was a minor lockup issue happening once or twice a month, but it happened several times today. I had already prepared a fix but wanted to wait and install it later with some new reports I've also been working on, but I went ahead and put it in today - so now you can see the new reports page, but they won't do anything yet - more importantly, those annoying lockups should be gone for good now - so today was a very unusual day - I hope the experience didn't scare anyone away. As I was writing up this third outage description, I decided to make it a new page with the latest info at the bottom to make it easier to add new info to.
Oct 4, 2011
I made some good progress on the stability issue. It isn't possible to prove there are no other problems, but I squashed one nasty bug that was bringing down the system with uncomfortable frequency. The trouble was related to the EditItem page's image upload feature - whenever an image upload was abandoned (often because the file was too large), it would never release the thread servicing it, so eventually it would use them all and have none left for the rest of the system. This was due to a bug in the image resizing system library I was using - I found a better one that doesn't suffer from this problem. I added the image upload feature over a year ago, but we just haven't had as many folks using it to cause this problem I guess. You can stop reading now if you aren't interested in the technical reasons behind this problem.
I also trimmed the size limit down to 7 meg per image, but I may try to raise that again now that I know this was the problem. I also added a wicket version id on the About page for any technical folks who may be curious about that (I did a lot of experimenting with versions before I found this system library that had changed). By the way, the fix I referred to above was only a symptom fix (inability for subsequent pages to obtain a lock). The root cause was the stuck thread holding the lock (because it waiting forever in the image resize routine).
Oct 13, 2011
I had a 'eureka' moment today when I made the connection between folks asking about reports and the frequency of these lock ups - I also learned a nice diagnostic trick 'kill -3' to initiate a stack dump during a lockup - this finally led me to the root cause I've been looking for, which makes total sense in hindsight but had for some reason eluded me for a while - it's all due to a connection pool leak whenever someone runs a report - we get exactly 20 runs, then kaboom on the 21st one. Very easy fix. I hesitate to declare victory prematurely, but this time I really think I got it taken care of and we should be back to the usual level of stability you can depend on. Whew! - Just in time for auction season!
Oct 17, 2011
After several days with no problems, I added the graph at the top of this page to the home page also - this is an independant website monitoring service that will send me an email within 5 minutes if the site ever goes down again. Hopefully this will be my last entry here for a long time.
We had a brief power outage today from approx 1:08pm-2:43pm - the automatic notification system alerted me immediately, and the system restarted without intervention when power was restored. I recommend sites such as http://www.downforeveryoneorjustme.com/ if you have trouble accessing this or any other website.
July 1, 2012
We had a big storm system pass thru the midwest Friday, knocking out power for several days now. American Electric power has given general estimates of up to 3 more days before 90% of customers have service restored.
Update: as of 7/3 power was restored, but still no internet service.
Update: another (less severe) storm caused shorter power outages, and internet service restored 7/5.
Besides the power outage, I also fixed a BIOS setting that left the system powered off after an extended outage. Also had to repeat some network driver related system config that was lost when I applied some security patches. (Ubuntu kernel didn't recognize proper R8168 driver for network card). Tested full reboot OK after fix.
System fully functional again 7/6 at 8PM EDT.