Notsuki Ame

Other thread is so clogged, felt like a new announcement would be more visible. This thread is locked and purely for announcement only.

 

Bot Mitigation

I will be implementing some bot mitigation techniques. This will be the first of changes along with different reader. From the brainstorms, I've learned a few things that may assist in curbing bots, in addition what I thought of implementing. Some of it will be a bit more of an enterprise solution, some of it will be custom solutions, and some of it will just be basic changes that make crawling more of a hassle.

 

These will be added over a long period of time. Almost all of these are machine targeting machine. And you won't even see it.

 

Changes to Reader

To assist with the bot mitigation and reduction of resource usage, the reader will be changed a bit. It will look mostly the same, some might not even notice. But it is coded in a way that'll help me track the abuses more as well as making it lighter weight. So, the users should be able to hit next page faster too. Yay! This is already coded and being tested.

 

Unfortunately, this comes with new URL scheme. Old URLs will be shut down once new is fully operational and without issues.

 

Also, it will now require javascript & cookies to function. So... to the <1% that has it off. Sorry.

 

Membership Requirement

This has been a source of lot of controversy, but I do still find it almost certain it will happen. Yes, some users will leave. Ad revenue may go down, but so will operating costs. They always move together. I foresee no financial concern from this change. On top of that, kenshin has offered more help that's acting as a safety net in terms of cost issues. And if that means I get to have more horsepower to serve the members that care, I see no real concern. I believe what made Batoto into the giant that we have grown into are the contributors. And by contributors, I mean from people who's uploaded thousands of chapters and to people who's made a single comment about their thoughts. Contents are what makes this site, and they already are members.

 

There are no plans to close membership nor anything like a paywall. You can register anytime you want, and become part of this community.

 

Membership requirement will not be a sudden switch. It will be eased in. And what requires membership will slowly grow. The slow pace will give me the numbers to confirm/break my expectations. And that is from where I'll ultimately draw the final choice, not from guesses or what might be. I can't see the future, nor can anyone else from thin air.

 

Uploader's Private Direct Link

This is one change of direction I realized may be helpful after reading the feedbacks in various places. The uploader, optionally can generate a second URL to the chapter. It will not be mentioned anywhere on this site, but people coming to that URL from the source site won't need membership to view. That way, we can still provide a reliable way for uploaders to share without affecting the anti-crawl of our system.

 

I will likely add this sometime while membership requirement is being eased in. Hopefully in early stage.

 

Why are Changes Being Made Now?

Quite a lot seemed confused about this. I do apologize for lack of clarity initially. I've said before, but complete prevention of copying Batoto isn't the goal. It's just not possible. I know that some of them have humans to copy from Batoto in rare occasions that I troll them with fake contents. Goal is to reduce it.

 

Over the years of maintaining Batoto, I didn't do it alone. I have hired other professionals in the past occasionally to help. And their first reaction to my server is always that I'm getting DDoS'ed. If that were true, this is the longest DDoS in history of the internet. (They do eventually know that it's not) We just get crawled that much. We have to get it down to a reasonable level where it's manageable. This, is a viable goal.

 

It's not that Batoto's going to die tomorrow without making the change. Or a month or etc. It's an ever growing problem and has to be tackled sometime. Doing nothing is to just shut down.

 

After

Hopefully focus can be sent back to bunch of stuff that's like 80% done but never finalized... Like follows overhaul. Flattr/changetip that was started like years ago... So much in queue.

 

Self-Published Titles

None of this applies to self published titles. Except automatic bot mitigation.

I write this announcement about becoming private, meaning that only members will be able to browse this site (or a portion of it).
 
I post this with quite a bit of reluctance because it's something I wanted to avoid. Since pretty much the beginning, there have been requests to make this site private or to increase security of this site so that it doesn't become simply the source for all the other manga aggregates. But I have always said no because I envisioned this site being as open as possible and available to as many as possible. That principle is why, for basic reading, this site even runs without javascript, flash, or pretty much anything beyond most basic requirements--something few sites do today. These two goals of openness and security are both something we want to achieve but they are contradictory to the other and in the end I've chosen ease of use for the users. Times are changing though and it's getting increasingly difficult to maintain this site while being as open as it is now.
 
Currently, Batoto is undoubtedly the go-to source for hundreds other aggregates. Changes and additions made on this site gets automatically updated elsewhere, like this love of ponies (if that link stops working). People uploading here did not intend for their works to be copied everywhere else. Complete prevention of a copy is not an achievable goal, but making it inconvenient--creating a deterrence is possible. And that is making contents viewable by members only.
 
The change kind of scares me to be honest. From an administrator's point of view. Batoto is very optimized to be viewed by guests because guests make up roughly 90% of the visitors--and it's much easier to optimize for guests as they require less dynamic actions. When they all become members, many of existing optimizations made over the years become useless. I don't know if our current system can handle the increased load. I don't know if decrease in bot load will be significant enough. I don't know if it'll make more members leave us. I don't know if ad revenues will be enough to support the system once it's changed. There's quite a lot of unknowns.
 
I feel the change is (most likely) inevitable. However, the question of how it changes is not certain yet. I'm open to hear suggestions and ideas on this.

 

Few things.

• We don't really care about SEO. Bare minimum of it is sufficient. It has never been a big thing for us. So, don't worry about that.
• The easier the hurdle to pass, the less useful it will be, but more user friendly.
• This change will break a lot of other sites and apps that rely on Batoto to function.
• Opting for partial private may also be possible. Like newly uploaded are visible for few days without registration.
• Objective: No more crawly crawly on this site.

-------------------------

 

Edits:

 

Well, this is certainly the hottest announcement topic.

 

I want to clarify something. I post this now and here not because I want to play police, ego or some stuff like that. If it was, I would've changed a long ago. I post this now, I change my objective as someone who maintains this site. I want to shave off a few million page hits a day. With great thanks to kenshin, our bandwidth costs don't increase that much with increased traffic. I still maintain my original image source nodes, so it's not a big shave off in cost, but it is completely manageable. What's biggest (always has been) cost is the HTML of this site, processing the pages that needs to be served. These run on farm of really beefy CPU servers with SSD and I'm currently looking to see if it's necessary to purchase another to handle the load. And one of my ad networks haven't paid me in 3 months. So I'm running on a red.

 

We have over 10,000 comics. And over 300,000 chapters. When 100 other crawlers think they need this content, some of it on a few minute basis (to check for new chapters)! Numbers REALLY add up. Humans don't do this. Because there is follows. You don't need to visit 10,000 comics just to see if there's something new.

 

There are number of anti-crawl features on this site already. All of which I tried to make that doesn't hinder normal users at all. It has caught a few real people using download scripts too. But it's insufficient; it's too lightweight. Pretty much since year 1 of Batoto, other crawlers have been using IP distributed crawls. Without further tracking tools for me, they're just not possible to track.

 

Hitting some of the new chapters are less of a concern. It's the deep crawl that concerns me.

 

-------------------------

 

Edit 2:

 

For those saying just don't do it. The alternative is to shut down. Crawling is non-sustainable. Have to do something.

This?

I think so too! I think it’s rare to have great manga with great endings.. Do mangaka enjoy so much ending their work like shit so much? Though, this is just my point of view, not everyone’s.

Yeah, I often wonder what the creators feel when they let their work descend into mediocrity (or even start as mediocre works) because of sloppy editing, forcing them to pander to the common denominator...

*weeps for Gamaran, Fairy Tail & Kenichi, among many others*

My favorite completed manga are 666 Satan and Sundome.

Ooo, Sundome! A truly unique manga.

I thought the way the ending was handled was fascinating, I've never seen everything get resolved with so few things being displayed outright.

Just an hour in the future? Depends for whom~ You might even be in the past for some :3
Haha nice to meet you and Welcome to Batoto!
Enjoy your stay over here~