himan001

I've been getting notices from various users about fake Batoto sites that's out there.

It looks exactly like us (pretty sure direct copy of everything), but with their own ads.

DO NOT LOGIN UNDER ANY CIRCUMSTANCE on the fake site. They're stealing your id/passwords. If you end up on that site for any reason, LEAVE IMMEDIATELY.

If you ever logged in to one of those sites before, change your password on EVERY SITE that uses the same password, not just here.

Our domain is bato.to. Anything else is fake.

Other thread is so clogged, felt like a new announcement would be more visible. This thread is locked and purely for announcement only.

Bot Mitigation

I will be implementing some bot mitigation techniques. This will be the first of changes along with different reader. From the brainstorms, I've learned a few things that may assist in curbing bots, in addition what I thought of implementing. Some of it will be a bit more of an enterprise solution, some of it will be custom solutions, and some of it will just be basic changes that make crawling more of a hassle.

These will be added over a long period of time. Almost all of these are machine targeting machine. And you won't even see it.

Changes to Reader

To assist with the bot mitigation and reduction of resource usage, the reader will be changed a bit. It will look mostly the same, some might not even notice. But it is coded in a way that'll help me track the abuses more as well as making it lighter weight. So, the users should be able to hit next page faster too. Yay! This is already coded and being tested.

Unfortunately, this comes with new URL scheme. Old URLs will be shut down once new is fully operational and without issues.

Also, it will now require javascript & cookies to function. So... to the <1% that has it off. Sorry.

Membership Requirement

This has been a source of lot of controversy, but I do still find it almost certain it will happen. Yes, some users will leave. Ad revenue may go down, but so will operating costs. They always move together. I foresee no financial concern from this change. On top of that, kenshin has offered more help that's acting as a safety net in terms of cost issues. And if that means I get to have more horsepower to serve the members that care, I see no real concern. I believe what made Batoto into the giant that we have grown into are the contributors. And by contributors, I mean from people who's uploaded thousands of chapters and to people who's made a single comment about their thoughts. Contents are what makes this site, and they already are members.

There are no plans to close membership nor anything like a paywall. You can register anytime you want, and become part of this community.

Membership requirement will not be a sudden switch. It will be eased in. And what requires membership will slowly grow. The slow pace will give me the numbers to confirm/break my expectations. And that is from where I'll ultimately draw the final choice, not from guesses or what might be. I can't see the future, nor can anyone else from thin air.

Uploader's Private Direct Link

This is one change of direction I realized may be helpful after reading the feedbacks in various places. The uploader, optionally can generate a second URL to the chapter. It will not be mentioned anywhere on this site, but people coming to that URL from the source site won't need membership to view. That way, we can still provide a reliable way for uploaders to share without affecting the anti-crawl of our system.

I will likely add this sometime while membership requirement is being eased in. Hopefully in early stage.

Why are Changes Being Made Now?

Quite a lot seemed confused about this. I do apologize for lack of clarity initially. I've said before, but complete prevention of copying Batoto isn't the goal. It's just not possible. I know that some of them have humans to copy from Batoto in rare occasions that I troll them with fake contents. Goal is to reduce it.

Over the years of maintaining Batoto, I didn't do it alone. I have hired other professionals in the past occasionally to help. And their first reaction to my server is always that I'm getting DDoS'ed. If that were true, this is the longest DDoS in history of the internet. (They do eventually know that it's not) We just get crawled that much. We have to get it down to a reasonable level where it's manageable. This, is a viable goal.

It's not that Batoto's going to die tomorrow without making the change. Or a month or etc. It's an ever growing problem and has to be tackled sometime. Doing nothing is to just shut down.

After

Hopefully focus can be sent back to bunch of stuff that's like 80% done but never finalized... Like follows overhaul. Flattr/changetip that was started like years ago... So much in queue.

Self-Published Titles

None of this applies to self published titles. Except automatic bot mitigation.

I write this announcement about becoming private, meaning that only members will be able to browse this site (or a portion of it).
 
I post this with quite a bit of reluctance because it's something I wanted to avoid. Since pretty much the beginning, there have been requests to make this site private or to increase security of this site so that it doesn't become simply the source for all the other manga aggregates. But I have always said no because I envisioned this site being as open as possible and available to as many as possible. That principle is why, for basic reading, this site even runs without javascript, flash, or pretty much anything beyond most basic requirements--something few sites do today. These two goals of openness and security are both something we want to achieve but they are contradictory to the other and in the end I've chosen ease of use for the users. Times are changing though and it's getting increasingly difficult to maintain this site while being as open as it is now.
 
Currently, Batoto is undoubtedly the go-to source for hundreds other aggregates. Changes and additions made on this site gets automatically updated elsewhere, like this love of ponies (if that link stops working). People uploading here did not intend for their works to be copied everywhere else. Complete prevention of a copy is not an achievable goal, but making it inconvenient--creating a deterrence is possible. And that is making contents viewable by members only.
 
The change kind of scares me to be honest. From an administrator's point of view. Batoto is very optimized to be viewed by guests because guests make up roughly 90% of the visitors--and it's much easier to optimize for guests as they require less dynamic actions. When they all become members, many of existing optimizations made over the years become useless. I don't know if our current system can handle the increased load. I don't know if decrease in bot load will be significant enough. I don't know if it'll make more members leave us. I don't know if ad revenues will be enough to support the system once it's changed. There's quite a lot of unknowns.
 
I feel the change is (most likely) inevitable. However, the question of how it changes is not certain yet. I'm open to hear suggestions and ideas on this.

Few things.

• We don't really care about SEO. Bare minimum of it is sufficient. It has never been a big thing for us. So, don't worry about that.
• The easier the hurdle to pass, the less useful it will be, but more user friendly.
• This change will break a lot of other sites and apps that rely on Batoto to function.
• Opting for partial private may also be possible. Like newly uploaded are visible for few days without registration.
• Objective: No more crawly crawly on this site.

-------------------------

Edits:

Well, this is certainly the hottest announcement topic.

I want to clarify something. I post this now and here not because I want to play police, ego or some stuff like that. If it was, I would've changed a long ago. I post this now, I change my objective as someone who maintains this site. I want to shave off a few million page hits a day. With great thanks to kenshin, our bandwidth costs don't increase that much with increased traffic. I still maintain my original image source nodes, so it's not a big shave off in cost, but it is completely manageable. What's biggest (always has been) cost is the HTML of this site, processing the pages that needs to be served. These run on farm of really beefy CPU servers with SSD and I'm currently looking to see if it's necessary to purchase another to handle the load. And one of my ad networks haven't paid me in 3 months. So I'm running on a red.

We have over 10,000 comics. And over 300,000 chapters. When 100 other crawlers think they need this content, some of it on a few minute basis (to check for new chapters)! Numbers REALLY add up. Humans don't do this. Because there is follows. You don't need to visit 10,000 comics just to see if there's something new.

There are number of anti-crawl features on this site already. All of which I tried to make that doesn't hinder normal users at all. It has caught a few real people using download scripts too. But it's insufficient; it's too lightweight. Pretty much since year 1 of Batoto, other crawlers have been using IP distributed crawls. Without further tracking tools for me, they're just not possible to track.

Hitting some of the new chapters are less of a concern. It's the deep crawl that concerns me.

-------------------------

Edit 2:

For those saying just don't do it. The alternative is to shut down. Crawling is non-sustainable. Have to do something.

... It seems to have targeted IE users.

Well... don't panic... but it appears Batoto has been hacked.

Google scanner reported that there is malware present on this site about half a day ago from this post date which caused the malware warning being displayed to anyone trying to visit the site which started the investigation.

An encrypted javascript injection (simply malware/virus) was found in Batoto's skin code, namely Deluxe. This is highly concerning since the admins are the only ones supposed to be the only one who should be able to access that. I cleaned out the infection, but later decided to nuke the entire skin as there may be more lingering pieces. Deluxe skin has been deleted and will not be restored. In place, our old skin, Sylo is back to default and Blood is still there as an option.

At this time, I am unable to be certain of the scope of the damage or the point of vulnerability. I am going to further investigate as to since when this virus was present later.

I'm not sure what the injected javascript does, because whenever I tried to access it, I either got an already suspended account it led to or a file not found page (404). But it is possible the not-found page is a disguise since it's possible to make the URL time gated and/or referrer gated to prevent accessing the same URL again later. At the same time, it's entirely possible the URL the virus tried to load really didn't exist and resulted in no action.

The virus itself seems to often hide itself, thus making detection difficult. On top of that, it seems very new (or recently altered) and does not exist in many anti-malware's database. That's why there were even notices by Google saying there's malware but found 0 pages with it. Almost no scanner I tried had detected it and almost gave up using scanners. It seems to have targeted IE users. I could not make the injected code appear using using firefox/chrome browser headers. It targeted users that came from a search engine. It was also invisible to standard googlebot.

During investigation, I also found out that the same kind of virus was previously present at another IPB site, completely independent of ours. And similarly, it is running version 3.4.6 (latest). So it seems most likely that this virus targets IPB sites and may be a zero-day exploit (an exploit that a patch does not exist for). I've sent little, but much info as I can to IPB as well in case it really is a zero-day attack vs IPB systems.

The biggest concern right now is that I still don't know exactly how the system was breached. If this really is an attack vs current version IPB system, I can't guarantee the virus won't return. If it was an exploit caused by the Deluxe skin, I have rid of it, but that's just a guess. I've been scouring the logs for last few hours... and besides the typical dozens attempts that failed (this is nothing specific to us, just life of sys admin), I really don't see anything special right now.

AS OF NOW

no unknown 3rd party items are being pulled during loading of Batoto pages. So google is no longer flagging us as suspicious.

If you see anything suspicious from this moment on, please report them immediately! If you don't have an account, you can email me: [email protected]

Security tips & Profiling

These kind of malware (javascript injected into sites) are typically are after putting their ads / redirecting you to somewhere else / installing virus on your computer. From the reports I've gotten, it doesn't seem like that's happening. But just as a tip... If this site, heck any site or anything, anywhere EVER tells you to install something, don't. Only time you want to install something is if you initiated it. If you went out to search and install that thing.

Also, if you'd like, Batoto is designed to be able to run without JS or flash. You can run this site with pure html & css only which has zero risk. But certain features will be inaccessible.

tl;dr

It looks like we're okay. But we might not be.

p.s. This incident is not related to the ads.