On Castine and its many tribulations

Castine is an imaginary country that once existed on the imaginary planet I prefer to call Ogieff. In fact, the imaginary planet doesn’t have an official name – it’s hosted at opengeofiction.net, which all the users call, simply, “OGF”. That initialism leads to my preferred name for the planet – just sound it out.

There is a real place called Castine – it’s a small town in Maine, USA. This is not that Castine.

I joined OGF in 2014, and Castine appeared and began evolving sometime in the year after that, I think – in 2015. I also became an admin on the opengeofiction.net website in that year.

During the period from 2015 to 2017, Castine became the locus of a kind of meta-proxy-war, where I used it as a stand-in for a never-ending argument I liked to have with my fellow OGF admins.

The issue in question was the rule about “verisimilitude”. I had long felt (and continue to feel) that OGF’s verisimilitude rule is a bad idea – it’s vague and impossible to enforce consistently. It has no objectivity. The principle is that mapping on the OGF world is supposed to be “realistic” in the sense that it eschews fantasy and sci-fi elements, and doesn’t contain cultural or cartographic artifacts that couldn’t reasonably exist in the real world. Hence, people who build 50 km bridges or tunnels are called out for violating verisimilitude, likewise more science-fictional elements like space elevators or fantasy elements like dens of dragons or nations of 1920’s-era talking sheep (all these examples really occurred at various times on the OGF planet).

Castine was (is) a borderline case of violating verisimilitude. Some users felt it violated the rule, others felt it was okay. My position was always something like: “since we can’t decide if this violates verisimilitude or not, but it’s really good mapping… c’cmon, people, let’s drop (or at least, fix) this stupid rule.

Of course, this was an unpopular stance. And in the long run, I lost the battle to remove or even alter the verisimilitude rule on Ogieff, and I made my peace with it.

One way that I made that peace with it, was to create my own, separate planet! In 2016, I started the planet Arhet as a kind of alternative project to Ogieff. By 2018, it had several active mappers and its own emerging community. The principle concept behind Arhet is to be a kind of “libertarian” reinterpretation of OGF. It has very few rules: no verisimilitude rule, no assigned territories, etc. And somewhat to my own surprise, it sorta kinda works. The key to it working, I reckon, is that unlike OGF, Arhet is not “open” to any and all comers. There’s an application process to join, and although I enforce almost no rules for the planet, I do stand firm that arguments or disagreements between users that escalate to my remit will simply result in immediate banning of all parties. That keeps everyone participating on best behavior, I guess.

The irony is that then, in 2021, I took over the hosting of the original opengeofiction.net. So now I host a little federation of two imaginary planets, Ogieff and Arhet, which have substantially overlapping user communities but having quite different rule systems. And I’m okay about that. I inevitably yield to my fellow admins, whose hard work and dedication to the project I admire, when it comes to matters of rules and judgements on Ogieff. But off to the side, I run Arhet singularly, and I insist on its fundamentally anarchic state.

In around 2020, the creator of Castine (Ramasham) was banned from Ogieff – ultimately for violating another, different rule: the rule prohibiting direct upload of data copied from OSM. OSM is OpenStreetMap, which is a map of the Real World™ in the same technological vein as our two imaginary planets. This is the so-called “slippy map” paradigm, originally popularized by mapquest and perfected and dominated by google maps. OSM runs on and supports a whole complex ecosystem of software that is all open source, as a kind of alternative to google maps, and that’s why it’s easy (uh, “easy” in a financial sense, not “easy” in a technical sense) for us to use the same software to run OGF and Arhet.

Anyway, there is (and there has always been) a rule prohibiting copying OSM data into OGF. Ostensibly this is motivated by paranoia about copyright violation, but in fact copyright has little to do with it, in my own estimation – there are easy ways to avoid issues around copyright as long as you follow along with OSM’s “attribution and re-use” rules. The real motivation for the prohibition is legitimate, though: on OGF, we want to discourage mappers from spamming the planet’s map with cut-n-paste copies of real-world places. It’s low effort geofiction and discourages creativity.

That said, when I set up Arhet I decided to also not enforce OGF’s “no real-world (OSM) data” rule. And indeed I myself played around with cutting and pasting some data from OSM, including an ephemeral instance of country I called “Lingit Aani” (this is Tlingit language) – a copy of the islands of Southeast Alaska but minus any nearby continent, as an open-ocean archipelago. I later deleted this, but there are multiple copy-the-real-world geofiction projects going on in Arhet, these days, including clones of Sakhalin Island (Siberia) and Romania’s Bucharest, and at least two Polands – perhaps more.

I guess Castine’s creator, Ramasham, had been doing some copy-pasting of OSM data to increase the detail and complexity of Castine’s cartography. Notably, this airport is a modified cut-n-paste copy of one in the real world, with only the names of things altered. And so Ramasham was banned from OGF. Rules are rules, and that “no copy from OSM” rule is actually probably the most common reason for mappers to be banned from the site.

Now we come to February of this year (2022). The admin team at OGF, moving to “clean up” various abandoned territories around our (imaginary) globe, decided finally to delete Castine once and for all. And I had a moment of deep sadness and regret. Despite my having leveraged Castine back in 2016 as part of my proxy war with the other admins over the verisimilitude rule, in fact I really, really like Castine.

From a technical standpoint, Ramasham was at best a mediocre mapper. But the imaginary country is full of cartographic whimsy and playfulness, the naming is thorough and inventive and culturally intriguing, and the detail in some parts is quite incredible. I thought it was worth preserving.

So I considered: Ramasham’s ban from OGF was for violating the “No OSM data” rule; if there were any other issues with Castine, they were issues with the “verisimilitude” rule; so… hey – Arhet doesn’t have those rules!

The solution was obvious. I decided I’d move Castine to Arhet. And even more conveniently, the exact latitude and longitude of Castine’s old Ogieff location was open an unused on Arhet. I figured it should be quite easy to simply “cut-n-paste” the whole of Castine into Arhet.

Yikes! This turned out to be the far from the case – it was not easy. Not at all. Castine included almost 2 million distinct GIS objects: nodes, ways, relations. This was not trivial to simply cut, paste, and upload into the new site. And further, the data quality was quite poor, from a technical standpoint. Thousands of improperly stacked ways on shared nodes, hundreds of lazily-crafted or incomplete data relations, etc.

I have spent the last week in a kind of Alice-in-Wonderland nightmare of trying to rescue Castine and upload it to the Arhet planet. I think that as of this morning, that I have succeeded, but not before almost destroying the Arhet server altogether in the process.

Without going into a lot of detail, it seems that there were a couple of relations (a technical term in this case for a type of data object used in OSM GIS software) that were apparently so badly constructed that they broke the server’s database. Since I had to do a kind of trial-and-error search to finally identify these objects, it took a very long time. I’d upload some subset of the full Castine dataset, and watch to see if the database crashed or not. If it didn’t, fine, I’d try another set of data. If it crashed, I’d have to go back to the last backup of the server, restore it, and try again. I think I did a backup-restore cycle maybe 12 or 14 times over the last week on the Arhet server. It was painful, and tedious, and immensely frustrating.

The crash-provoking objects in question are puzzling. I still don’t understand why they crash the database. And given my difficulties in identifying them (and surviving them – see below), I probably won’t spend time, any time soon, trying to figure them out. They are “Giant Chessboards” – three of them. Interestingly, Castine also has other “Giant Chessboards” (e.g. here) that do not cause any kind of data problem. They are apparently implemented differently, in their details.

The problem was compounded yesterday, when, much to my shocked dismay, the server-level backup-restore functionality offered by my hosting provider, Linode – that I’d been so repeatedly abusing – suddenly and inexplicably failed to work.

So for a day (yesterday) the world was Arhetless. The server was down. I was in a panic because it seemed I’d have to fully rebuild the server from scratch. And it was only pure luck that I even had a copy of the map data, because I was still running a kludgey render engine (map drawing process) for Arhet on a different machine.

I wrangled with tech support at Linode, and they finally held my hand (or was it that they held my server’s hand?) through a successful if stressful restoration of the server’s image.

Let’s just say, these days Castine now has a quite colorful meta-history.

I reached out to the creator of Castine, sending an email to the address on record at OGF, announcing its restoration in Arhet. I would absolutely welcome and be pleased if that person would come back and take up work on the country, again – they won’t be constrained by the rules and regulations on Ogieff. Unfortunately I haven’t heard back. I speculate that there might be some bitterness about the whole business of having been first praised and then banned, back a few years ago.

The link to Castine-in-Arhet is here:

https://arhet.rent-a-planet.com/relation/10996

Please feel free to explore. I decided not to bother with adding extensive screenshots for this blog post – the point of having the Castine map hosted on the server is that you can explore easily directly on the website.

Happy mapping.

Music to view geofiction to: Dawg Yawp, “Lost At Sea.”

On Mass Deletions in OGF

This morning, I posted this in response to a query on the User Diaries at OpenGeofiction:

General advice for all users:

When deleting large numbers of objects, please be careful. This is not a use-case that the OSM software is designed to handle (think about it – mass deletions are NOT common on OSM). Divide up your deletions to cover small numbers of objects (<1000) and small areas (so if something goes wrong you don’t mess up large areas).

I called attention to the post in the discord channel (“OGFC”), and so I decided to rant onward and provide some additional clarification and thoughts. I’ve distilled those, without much editing, below.

To the above, I would add that in my own practice, I have another step I take with respect to deletions. I almost never delete relations. You’ll notice in JOSM, it actually tries to warn you that “this is rarely necessary” when you try to delete a relation, and I think that’s a solid point. What you can do with relations you don’t need any more is repurpose them. That means you delete the members of the relation, and all of its tags, and then use it the next time you need a new relation for something.

user: What would be the potential harm of deleting relations?

Well I suspect that sometimes, a deleted relation can end up as a “ghost” object, or a “stranded” object, in the render database. Meaning it’s truly been deleted from the API database (the main rails port) but that deletion fails to transfer correctly to the render database (which stores things in a different format, and doesn’t actually have “relations” at all, but rather transforms a relation them into a special type of “closed way” I think)

By repurposing a relation instead of deleting it, you guarantee the render database will have to reclassify the linked object(s).

If you just delete it, the render database might not handle the disappeared relation correctly, and just leave the no-longer-linked objects lying around.

This is just speculation, because I don’t really understand it well. But just based on observed behavior

In general, I think that the OSM software is not designed to handle mass deletions. That’s the key point.

Because in OSM, who goes around deleting large number of objects? The real world doesn’t work that way.

You re-tag things, yes. You might move things slightly, adjusting position to improve accuracy. But things in OSM don’t “disappear”

Or if they do, they do so in very, very small sets. A building gets knocked down. A road gets torn out.

One object at a time.

So it seems very plausible to me that the OSM software actually hasn’t been designed to handle mass deletions, and hasn’t been tested for integrity in dealing with mass deletions.

For years now, I’ve been telling people, when you decide to rearrange your territory (and what geofictician doesn’t sometimes decide to rearrange their territory?!) … it’s better to move things than to delete/re-create things.

This prevents “ghosts” in the render.

In the long run, I suspect that we’ll have to just “reboot” the render now and then. But I’m not going to do it very often. (I mean “reboot” as in delete and re-create from zero, not “reboot” as in turn the machine off and on again – that happens every night).

I’d welcome comments on this rambling, vaguely rant-like discourse, for those who are knowledgeable about how the OSM apidb-to-renderdb pipeline works.

Music to make deletions to: 10cm, “오늘밤에.”

The Terrible mysql Crash of 2021

I still don’t know how it happened. I somewhat suspect I got hacked, somehow … I found strange and unexpected Chinese IP addresses in my mysql error log. But I don’t understand mysql back end or admin well enough to know for sure what was going on.

I was able to restore a full-server backup to a new server instance, and have re-enabled the mysql-driven websites (my 2 blogs, my wiki, etc.) on the new instance. Meanwhile, I somewhat stupidly reactivated the non-mysql website (the geofictician OSM-style mapping site, the so-called “rails port”) on the old server instance. The consequence of that is that I am now stuck with a two-server configuration where I had a single server configuration before. I think in the long run I’ll want to isolate ALL my mysql-based sites to a single server, and ALL my non-mysql-based sites to another single server. That’s going to take a lot of shuffling things around, which is not trivial.

For now this blog (and my other blog) seems healthy and up-and-running, again.

There may be more downtime ahead as I try to reconfigure things more logically, however.

Music to do sysadmin drudgery by: Talking Heads, “Found A Job.”

Too many projects, not enough motivation.

There are many things I could be working on, geofiction-wise. I was chatting on the OGF Central discord channel a few days ago, and managed to enumerate 9 different ongoing geofiction map-drawing projects, all essentially unrelated.

  • OGF / Makaska
  • OGF / Tárrases+Mahhal
  • OGF / Ardisphere
  • Arhet / Deadlands+Hellbridge
  • Arhet / Rasfsayan
  • JOSM Only / Lekista
  • JOSM Only / Tsiqeye+Preye+Domeye+Sekeye (4 continents of a planet)
  • JOSM Only / Bofobunda+Zhebeyem
  • JOSM Only / Senhar

All of these above-listed projects are basically not moving forward at all.

Meanwhile, I also feel that I should be working on things like upgrading and/or completing the deployment of my map server (currently called Arhet). I also wish I had the energy to develop my expertise in the realm of getting contours working on Arhet, as they do on OGF. I won’t be happy with my own server environment until I’ve solved that.

But I have zero motivation, lately.

Anyone have any motivation to spare?

Music to motivate by: Taylor Swift, “The Man.”

Not much mapping

Really, I’ve just been busy with other things. I have been doing very little geofiction – exactly zero on OGF for the past month, and only minimal work at Arhet on my own server. I’ve been quite busy with real life.

And just now, I’m traveling. Here is a perhaps-geofiction-inspiring picture of Astoria, Oregon, seen from the Washington side of the Columbia River.

picture
The long Astoria Bridge across the mouth of the Columbia River, looking south at the town of Astoria, Oregon.

Music to fail to map by: Café Tacuba, “El Aparato.”

A year later

This blog is one year old today. I founded it on Saint Patrick’s Day, last year. That’s why there’s that little shamrock on the first entry.

I really haven’t posted as much as I intended, here, over the last year.

My life underwent such huge changes, mostly unexpected. I ended my 11 year residency in South Korea and moved back to Southeast Alaska. I’m still in a bit of transition in terms of career, and meanwhile living off my savings.

I took a break from the OGF admin team last summer, then worked really hard the last few months. I have become very frustrated with trying to do admin on that site. Indeed, I have become deeply disillusioned – mostly with myself, and my inability to maintain a charitable and good-willed mindset in dealing with a never-ending onslaught of faceless trolls and juvenile idiots. I’d rather cope with a classroom of unruly 7th graders.

In a few days, I’ll be traveling to my mother’s in Queensland for a few weeks: a long crossing of the Pacific. I’ll be constrained by obligations to relatives, so I’m taking a leave-of-absence from OGF and geofiction. I have resigned the admin position permanently. It will be hard to let go, but I feel I must do so for my own peace of mind.

Music to map by: Olga Bell, “Пермский Край.”

Subway Philosophy

Someday, I will return to work on my great metropolis, Villa Constitución. And when that day comes, I shall take on the huge project of refactoring the complex subway system I designed.

When designing subways, one should have a philosophy of subways in mind. Here is an essay every subway designer must read: “Stoppism: Retrospects and Prospects“.*

*Footnote for the dense: the linked article is satire – a gorgeous, brilliant joke.

Music to design subways by: Silvio Rodríguez, “Santiago de Chile.”

admin blues

<rant>

It’s all pretty depressing.

I try to be a competent and fair and innovative admin on OpenGeofiction.

Half the users hate me – I know this for an actual fact, because I see what gets said on the OGF unofficial discord channel.

And now I’m feuding with the “boss” too. I can’t win – I’m stuck in the middle. I’m not paid for this. So why am I doing it?

Perhaps I should go back to trying to build my own geofiction server and forget this. Although I derive a lot of motivation and inspiration from the OGF community, trying to be an engaged and active member feels like more suffering than benefit, some days. I would do better to not try to change or “fix” things, but that’s not in my character.

I don’t know if the creator of OGF and I really share much in terms of vision. To initial appearances, he seems committed to the “open-” part of the name, and to open source projects and concepts. Yet upon further examination, he seems utterly uninterested in trying to go anywhere toward working out a more scalable and/or sustainable governance model for the site. And for any sizable internet community (or real community for that matter), governance is actually important. So in the end, it’s just a personal fiefdom. I can feel sympathetic to that… – that’s probably how I would set my own site up. But then, what’s the “open” about? Is it just because he used the OSM stack? It feels like false advertising: “Bait and switch.”

This is just a rant.

</rant>

Music to admin by: Robbie Fulks, “America Is A Hard Religion.”

Not so geoficticiany, are we?

I have become rather obsessively immersed in a non-server-related, non-geofiction-related project. Such is my nature. I’m easily distracted by new projects.

As a result, though I still load these new geofictician sites and the OGF sites daily, and even comment occasionally or tweak something here and there, I haven’t really been doing much.

I’ll get back to this soon enough.

I did spend a few hours building Mahhalian contours, the other day, with a mind on fleshing out the new, leaner, smaller, faux Mahhal-for-OGF (because the “real” Mahhal will be a separate planet file on this here server thing, eventually).

Music to map by: 매드 클라운, “콩.”

Some weeks…

And then, some weeks, I don’t get much done.

I started working on trying to customize my Rails Port (the main “copy” of the OpenStreetMap slippy map), and got very bogged down in the fact that the OpenStreetMap Rails Port is highly complex software written in a language and using an architecture unfamiliar to me: the infamous “Ruby on Rails.”

I dislike the way that the actual name “OpenStreetMap” is hard-coded throughout all the little modules. It seems like a poor application design practice, especially for an opensource project. One area where the name proliferates is in all the internationalization files. So I started wondering how hard it might be to get all these internationalization files to be more “generic.” The answer: pretty hard, at least for me.

I’ve wandered off down a digressive passage where I’m learning about software internationalization under the Ruby on Rails paradigm, but I’m undecided how I want to handle this. Do I want to try to solve it the “right way”? Or just kludge it (most likely by deleting all the internationalization files except perhaps English, Spanish, and Korean)?

Meanwhile I have also got pulled away by some non-computer, non-geofiction projects.

So… not much to report, this week – nothing mapped, nothing coded, nothing configured.

Music to map by: Sergei Rachmaninoff, “Piano Concerto No. 2.”