Please see the latest upgrades from the voting system:

• InSanCen (from CitizenDonatorPlus to ArchitectDonatorPlus)

Congratz and thanks to all voters!

Disclaimer: This is just a small rant about what I worked on the last 2 days. If you are busy, no need to read.

Ok, so you all hopefully read yesterdays blogpost in which I announced to move to chunk-sized maps. Well actually that was wrongly put, since chunks are only 16×16 blocks large, I actually meant to move to file-sized maps, and one file covers 512×512 blocks.

And the whole thing fell apart the further I progressed with it. First of all, all went great. I managed to find a nice and simple way to place all the generated files on the page with simple CSS and little code. Life was good. Then I wanted to move on to do the same for the heatmaps. The issue is that I had to then render the heatmap for each 512×512 block of each map.

But everything was different looking all of a sudden. I thought at first that the method that I used to translate in-game coordinates (-1000 -> + 1000 around a center) to screen coordinates (0 in the top corner to 2000 in the left/bottom) did not work for single files since the coordinates there start anywhere in the map. Working on that one for 2-3 hours I finally figured out that if you calculate a heatmap for a single 512-sized block, the spot with 1 hit would have the same color intensity as for another block with 100 hits, if that was simply the maximum for each block. Only if you generated a whole world at once,  you would be able to see where in the world the highest hits were. Ideally we should render ALL worlds into one file just to see where the balance is between worlds. So I found that I had to assemble the files of each map back together to a single file in order to create a heatmap. On top of that, since we are having individual lot images on each user’s page, we needed a whole world map. Otherwise, if a lot is split between two files, we would have issues there.

So I tried to make a program that would re-assemble the file-based images into one single image. For that you scan a world directory, list up all the files and tell a small program to arrange them in a 10×10 pattern That worked for some worlds but did not work for others. The issue was that we had more worldfiles per world than 10 by 10. Some rows or lines in the grid would have more files outside the borders. Teleporting and other issues sometimes generate single-block items outside of the borders and those made the map assembly a mess. So I had to first go to each world, check and remove excessive files, then make a list of the good files to make sure no unwanted files messed up everything. Now I was able to have complete maps, crop those again since the world as we set the border does not end at the file end either, render heat maps out of them, cut it again in lot-sized pieces for the user pages, and done.

So in the end we are back to single-file 2D maps, but they still use the better rendering engine, and we have heatmaps again. Yay!

I have been working so far with a 2D map engine (c10t) which has become outdated with 1.7.2. It was already outdated with 1.6.4, I had to install a whole host of complicated programs, compile the source myself etc to get it to work, but I have given up now. The new renderer that I am using is working fine, and I can make new blocks appear on the map (the ones that are currently pink in the new empire map) by simply editing a text file. It also looks much better.

The difference is that the new renderer does not create one single file, but one per chunk. I have to use another program then to re-assemble the file, which works only if the file has an exact known size & dimension, which is a bit of an issue because I have to do trial-and error for each map that I want to make with this tool. If due to some error a chunk is created outside the map (this happens sometimes), the whole system breaks down and I have to trim the map of excessive chunks, and re-generate it. It’s annoying.

So the current 2D map is generated by having a background image (the one map image) and overlay it with the lot squares. What I will do now is to create chunk squares, have background images in them from the new map function and overlay those with the lot squares. I hope I find an elegant way to do this. The good thing will be that there will not anymore be a 3MB file in a single download like for the kingdom but instead several smaller files to download. This will speed things up a bit hopefully, and I can skip the map assembly tool, so there will be less code to maintain. You will know I managed to pull that off once all other maps look like the empire map.

I have now tested for the first time if it is working fine to take the old empire and fill all empty lots with data from the new empire. From what I can see this worked fine, so I generated a map from it. You can now see the new empire map with the currently existing lots. on the 2D map.

I have now installed the latest build of 1.7.2 and for a test installed all plugins that we are currently running. Of course some of them failed (such as minecart mania, mob disguises and a couple small ones), but none of the important ones that we need to run the server. What does that mean?

First of all, the fact that plugins do not crash during the start does not mean that they work 100%. There will be more testing needed for that. I assume that the forums of the individual plugins will soon be filled with comments in case something breaks.

Regarding the upgrade, I want to wait some time until there is some form of stabilization in the bukkit development process. As of now, there are frequent new builds, and while there are right now no open critical bugs, that does not mean that there are no new ones coming. I want to see how often new developer releases are being added, wait until it slows down and at least wait until this weekend with the upgrade. Also, many plugins that deal with blocks currently will not work properly with the new blocks of 1.7.2. So they will disappear into a void in case we try to trade them, and sometimes those new blocks also fail for protection against griefing.

Since the upgrade will involve the reset of the darklands (thanks to tbrewda who found a seed with mesa canyons at spawn!) and all the empire lot resets, I do not want to make an upgrade and then realize later that a bug messed something up. With this upgrade, it’s a major work to manage it, and I do not want to have to do it twice.

But I guess the big message is the following: If you want to reset your lot or pick a new one, now is the time to look at the map make a choice. I will introduce a process where you can submit requests for a different lot. And in this case it would be also the time to dismantle your belongings and stock them into chests.

I have tried around with a couple of seeds for the new empire while working with the first 1.7.2 dev builds and found one that has very little ocean and a nice balance of biomes. Please see the new map with the overlay of the current lots here. The pink stuff are trees and flowers that are new in 1.7.2, the renderer cannot handle them properly. The brown area around lot C25 is just dirt without grass.

Regarding the lot resets:

1. You will be able to reset any of your lots in empire. It will then look like the lots on the new map. You will lose ALL items on the reset lots, chests and so on. I will allow additional deposit storage during the reset time to help people move who only have one lot.
2. You will be able to chose a completely new lot (on the new map, has to be empty, not the old map). If 2 people want the same lot, we will make a clear hierarchy of choice by rank, donators included. That means that a Master will have their pick above Designers, and DesignerDonator above Designers and so on. If 2 people of the same rank want the same lot, we roll the dice.
3. You will be able to simply keep your lot and nothing will change otherwise.
4. All empty lots will reset too.
5. Strongholds and other locations with special mob spawn patterns (please read this here for more info) coming from the old/current empire will change the spawning to normal terrain features. Dungeons from the new empire will maintain their spawning behavior.
6. Yes, we will have breaks in the world and strange cut-off mountains etc, but we have those already now. Look at Lot EMP_S14 for example. We will not be able to avoid this. We can only allow people to reset their lot on world generator changes to try to reduce the impact.
7. No, I do not know yet when we will update. I will tell you when I am working on the upgrade. I will do so as soon as there is some more stable build of bukkit and when i know that all the important plugins are working.

Bukkit (the system, built on the normal (vanilla) minecraft server, that allows all our plugins and non-vanilla stuff to run) has released the first developer builds for 1.7.2. So what is keeping us from upgrading?

Well, as you can see in the graphic, there are still tons of bugs in it. The red line is the number of reported bugs, the green one is the resolved amount of bugs. Since the first dev version was released 1 day ago, people have been reporting bugs that need to be fixed until the first stable version will be released. The issue for us is that plugin developers need to fix their plugins to work with the new version, but since it will still change a lot (because they need to fix all those bugs), it does not really make sense to start working on the plugins before the biggest of them are fixed. So we have to first wait for the biggest bugs in Bukkit to be fixed, then for the plugin authors to fix their plugins. Please stand by and be patient.

In the meantime I will see what plugins still work and if I can try again to make a new map for the empire. So as you can see, I am frequently monitoring the situation so that we can upgrade as soon as possible.

I have found a module that lets you generate a heatmap from data and combined it with the previously mentioned location data captured during lag.

I also added a new link to the 2D maps where one can switch any map to a heatmap to see those new maps. Again, this is not a 1:1 locator for lag, specially not until we have at least 1-2 weeks of data. But then we can use it as a really good indicator where the issue is caused. The heat map will update with the other maps once a day.

I have now programmed a new function that records user locations when lag is high into a database. Then, when we render the 2D maps, it will count the amount of times users have been inside each lot at times of lag. This will be written into the map with a number.

So if you check out your lot on the 2D map, you might see a “Lag: 5” behind the lot name. Of course this does not necessarily mean something. It could mean that you are online whenever someone else causes lag. Or that you are just always online. My hope is that over time we will have quite accurate numbers about sources of lag in certain locations.

Regarding item frames:

Item frames are entities in minecraft code, that means that they are treated like mobs. If you have a lot of item frames in your building, to label your boxes for example, you create lag. I know that item frames are really cool to do that, and that they look great, but please keep in mind that this is creating an issue for the server. So please do me a favor and keep those to a limit. I do not want to limit them with code like we do for the mobs. But if we find out that the major lag locations is coming from the lots with many item frames, we will have to do something about it.

I would like to mention lag issues once again. The above illustration is one of the typical applications of redstone/minecarts/mobs that cause excessive and unnecessary lag (although it looks damn funny).

This is not supposed to be a rant, but more a reminder: Please be thoughtful of what you do with mobs, animals, redstone, minecarts etc. All those items are causing lag the more they are used. We have good times on the server with little lag, but we also have some times where the lag is very very heavy. As you might know, lag is measured in ticks per second (tps), and ideally we have 20tps. Many times however, we have less than 10. This will result in lag for all users. Often this is caused by automation mechanics that try to make the miner’s life easier.

If you are using any kind of redstone mechanics, I would like to encourage you to check the impact of those. This is best done when there are not too many people on the server. Check the “/lag” command before you switch on whatever you are doing, and then again while it is running. You can then judge the impact of your machines on the server. If it uses 10tps or more, you better take it down. If it is 5 tps or more, you better not run it when there are more than 10 people on the server.

We now have 2 PVP arenas built and ready. I placed mob spawners where requested. The whole thing works like this:

The upper arena (the nether-type one) has spawners for pigzombies and blazes. Otherwise no mobs spawn there. The lower arena has 2 spawners for zombies and 1 for pigzombies and zombies can spawn anywhere if it is dark enough. Otherwise no mobs spawn there. Both those arenas have PVP now ON.

What can be done next is a command system similar like the hunger games. This system can place people at one of a list of pre-defined spawn points.You guys have to decide how you want to manage those arenas. And you also have to tell me when they are finished built, so that I can take away the build rights.

The middle arena does not have PVP On yet, please tell me once you are ready to switch it on.

Please see the latest upgrades from the voting system:

• MrsBaxtersCat (from ArchitectDonatorPlus to DesignerDonatorPlus)

Congratz and thanks to all voters!

Please see the latest upgrades from the voting system:

• AZJaguar (from CitizenDonatorPlus to ArchitectDonatorPlus)
• RDStewart1992 (from Citizen to Architect)

Congratz and thanks to all voters!

I have now installed a new theme to the website that should make it easier to navigate the website with a mobile phone. The stad. theme was quite broken to serve mobile phones and this new system, while being not very exciting to look at on the phone, should give 99% of the functionality. Except for the forum where I strongly recommend to install tapatalk.

I wanted to outline here once how our backup system works right now so you know what is being done and what you can expect in case of a crash but also to get input in case someone with server maintenance experience wants to share possibilities for improvement.

1. We are running all limited-size world on a RAMdisk. Nether and Darklands are not. They are just too big for that. We are copying changes to the ramdisk to a save-folder with rsync to the normal disk every 5 minutes. If there is any issue (server crashes, system powers down), we should have 5 minutes loss maximum. On hardware restarts, the RAMDisk is built from these saves.
2. Every day once, I stop auto-saves on the MC server, copy all changed files with rsync to a mirror folder on a second hard drive and continue saving afterwards. This prevents us from issues where the main HD fails.
3. Every day we zip this mirror on the same 2ndary HD and store the archive in folders numbered 1-31 depending on the day of the month. Like that I keep always the last 31 days as a backup. Same happens with the databases.
4. Every week we zip the folders on top of that in folders numbered 1-52 depending on the number of the week. This way, in case we can track back an issue to originate further back than one month, we have weekly backups for one year. Same happens with the databases.
5. On top of that, I have a crashplan subscription which makes off-site backups to a cloud server of the daily databases and the weekly file mirrors of the drives. I had this subscription already to backup my photos to a secure location so no extra costs there. This is newly setup for minecraft since I did not know that I can install the crashplan client on a headless linux server.
6. Nether and darklands are neither mirrored nor backed up. The nether is now 2.6GB and the darklands 2.3GB which is together larger than all the rest of the files (5.4 GB). Given that we delete those from time to time, there was never a backup and there won’t be one.

So if anyone has some more suggestions on how to improve the system, I am happy for feedback. Finally I am not a learned sysadmin and still learn with every crash :)

Please see the latest upgrades from the voting system:

• lukecreative (from ArchitectDonator to DesignerDonator)

Congratz and thanks to all voters!

I also now resolved issues with teamspeak so the server is back online.