atgc

I write this announcement about becoming private, meaning that only members will be able to browse this site (or a portion of it).
 
I post this with quite a bit of reluctance because it's something I wanted to avoid. Since pretty much the beginning, there have been requests to make this site private or to increase security of this site so that it doesn't become simply the source for all the other manga aggregates. But I have always said no because I envisioned this site being as open as possible and available to as many as possible. That principle is why, for basic reading, this site even runs without javascript, flash, or pretty much anything beyond most basic requirements--something few sites do today. These two goals of openness and security are both something we want to achieve but they are contradictory to the other and in the end I've chosen ease of use for the users. Times are changing though and it's getting increasingly difficult to maintain this site while being as open as it is now.
 
Currently, Batoto is undoubtedly the go-to source for hundreds other aggregates. Changes and additions made on this site gets automatically updated elsewhere, like this love of ponies (if that link stops working). People uploading here did not intend for their works to be copied everywhere else. Complete prevention of a copy is not an achievable goal, but making it inconvenient--creating a deterrence is possible. And that is making contents viewable by members only.
 
The change kind of scares me to be honest. From an administrator's point of view. Batoto is very optimized to be viewed by guests because guests make up roughly 90% of the visitors--and it's much easier to optimize for guests as they require less dynamic actions. When they all become members, many of existing optimizations made over the years become useless. I don't know if our current system can handle the increased load. I don't know if decrease in bot load will be significant enough. I don't know if it'll make more members leave us. I don't know if ad revenues will be enough to support the system once it's changed. There's quite a lot of unknowns.
 
I feel the change is (most likely) inevitable. However, the question of how it changes is not certain yet. I'm open to hear suggestions and ideas on this.

Few things.

• We don't really care about SEO. Bare minimum of it is sufficient. It has never been a big thing for us. So, don't worry about that.
• The easier the hurdle to pass, the less useful it will be, but more user friendly.
• This change will break a lot of other sites and apps that rely on Batoto to function.
• Opting for partial private may also be possible. Like newly uploaded are visible for few days without registration.
• Objective: No more crawly crawly on this site.

-------------------------

Edits:

Well, this is certainly the hottest announcement topic.

I want to clarify something. I post this now and here not because I want to play police, ego or some stuff like that. If it was, I would've changed a long ago. I post this now, I change my objective as someone who maintains this site. I want to shave off a few million page hits a day. With great thanks to kenshin, our bandwidth costs don't increase that much with increased traffic. I still maintain my original image source nodes, so it's not a big shave off in cost, but it is completely manageable. What's biggest (always has been) cost is the HTML of this site, processing the pages that needs to be served. These run on farm of really beefy CPU servers with SSD and I'm currently looking to see if it's necessary to purchase another to handle the load. And one of my ad networks haven't paid me in 3 months. So I'm running on a red.

We have over 10,000 comics. And over 300,000 chapters. When 100 other crawlers think they need this content, some of it on a few minute basis (to check for new chapters)! Numbers REALLY add up. Humans don't do this. Because there is follows. You don't need to visit 10,000 comics just to see if there's something new.

There are number of anti-crawl features on this site already. All of which I tried to make that doesn't hinder normal users at all. It has caught a few real people using download scripts too. But it's insufficient; it's too lightweight. Pretty much since year 1 of Batoto, other crawlers have been using IP distributed crawls. Without further tracking tools for me, they're just not possible to track.

Hitting some of the new chapters are less of a concern. It's the deep crawl that concerns me.

-------------------------

Edit 2:

For those saying just don't do it. The alternative is to shut down. Crawling is non-sustainable. Have to do something.

So how do I hide the announcement sidebar?

You don't. I don't want people to miss things.

For a group called "easygoing", they sure work hard. xD