Work in progress & the hunt for errors

So some days ago, I tried to get a better grip on errors. Thing is, that normally, when you develop code on a linux box, you do so as a normal user. When something does not work, you need to log in as admin, browse to the folder where the logfiles are, see if you find the appropriate error (if there is one), log back as a normal user, change the code, hope it works, and if it does not, you do the same all over again.

It’s tedious. That’s when you are working on it. When something goes wrong while you are, let’s say, fast asleep, the only chance you have is that your users realize that something goes wrong and post a ticket. You then see the email, log in as admin, look for the error etc.

Important to say that there are a lot of so-called “warnings” and “notices” the system can store if you want which tell you if your code is not 100% written for safety in mind. With 2200 lines of code, I estimate that I have about 200-500 places in the code that create these notices – if I want. 80% of those are not really issues, but how would the system now. The logfile usually only stores the really important errors. The rest just disappears.

So I wanted to get some alerts when something really weird happened and set up an error reporting system that would send me an email when predictable things would go wrong. This would allow me to track errors that I could not quite pin down and send me a lot of information about variables, users and so on at the same time, which would not end up in the log files in the first place.

This worked fine, for 1-2 weeks. I then liked it so much that I used it regularly during writing code, so that I would get an email as a way if the code does what I want. The issue started when I made a mistake when iterating a larger amount of data and the system sent me 800 emails at once. Then, towards the evening, people started complaining that their registration emails would not arrive. I had no idea what happened for 2 days, until on the third day, I remembered that I have a 1000 email quota on the server which has been maxed out and thus also the actual important emails would not be sent anymore.

So I learned finally how to program my own error management system where I can decide what happens with errors instead of having the system write it into a logfile which is annoying to reach. So I had errors written into a database where I can access them from the website, group them by frequency and so see which ones are occuring the most and therefore should be fixed first. That allowed me also to make the system report all the warnings and notices, so that I could better audit the code.

But then I had the idea that I could get something more direct, more like email, but without the quota. I thought about IRC messages, but it’s a bit annoying since the server login etc takes time since the server is not on the local system. So I researched a bit more into instant messaging and found the XMPP protocol (formerly “jabber”), the same as used for google talk or facebook chat

Then I installed a XMPP chat server on the machine, taught the minecraft system how to send messages (that was the hard part) and now I made the system send me instant messages which do not have a quota, arrive instantly, even on my cellphone if I want to, AND I can send myself now all the notices and warnings to properly patch up the code.

So the last 24 hours or so I have been patching a LOT of small stuff in the code, which of course does not go smoothly all the time, but at least the server is code-wise in a much better situation, and getting much better and cleaned up substantially. And if there is an error, I will now about it instantly :-)

On top of that, we can theoretically have users use it. It would be like IRC, but fully locally authenticated, and people could get instant messages on their cellphones when they get an in-game message or their stuff is being sold. Of course this is all just distance future, but still nice as an option to have more feature freak :-)

7 thoughts on “Work in progress & the hunt for errors

  1. Nice,
    P.S. You’re someone who actually likes Linux :)

  2. XMPP/Jabber is a good way of getting notices, I confirm. Using it in-game would be quite good. That includes some important in-game events and player to player mail.

    Good luck!

    • yes! There are also plugins for chat that use xmpp instead of IRC. I am thingking about how to use that for our advantage.

  3. Seeing posts like this makes me really appreciate how much work you do for us Unc. This and how much I love the server and the community on it makes me want to donate more to keep it going. Its a shame more people don’t donate more to help you out. It would be nice to see, at least one time in the 2+ years I have been playing on this server, the donation graph in the plus. I know everyone can’t donate, but if most of the active players gave $10 or so ( the same as 2 packs of smokes, or 5 coffee’s) we would be in the plus in the blink of an eye. That is just my 2 cents. I know I have made posts like this before, but I feel strongly about it, so here is another one. (Hopefully the last ;) )

    • First things first: the amount and the quality of the job you do for us unc is something many people would not be prepared to/capable of doing for other people, I know I wouldn’t be. I am very grateful.

      Second, I agree with pagreifer completely about the donating part! And I also think that the perks this server gives for donating are pretty nice and useful, tho they don’t give you too much over non-donators. I’ve been donating from pretty much the beginning of my time here, and even tho I’m not very active at the moment, intend to keep donating. It’s a shame there are people who appreciate this server and the job unc does, but who don’t want to make it concrete by donating. I don’t wanna judge or lecture tho, but I do feel it’s not really right that we have been on minus all my time here. In the end the server is still free-to-play, and it’s good. I just hope I can make people think about donating.

  4. Thank you, Unc. I really appreciate all you do to keep this running smoothly. If there is any way I can help, even with more of a donation, please let me know.

Comments are closed.