Jump to content

Primary: Sky Slate Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate Marble
Secondary: Sky Slate Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate Marble
Pattern: Blank Waves Squares Notes Sharp Wood Rockface Leather Honey Vertical Triangles
Photo

Moving forward: Upcoming changes and schedules


  • This topic is locked This topic is locked
5 replies to this topic

#1
Grumpy

Grumpy

    RawR

  • Administrators
  • 4,078 posts
  • LocationHere of course!

*
POPULAR

Other thread is so clogged, felt like a new announcement would be more visible. This thread is locked and purely for announcement only.

 

Bot Mitigation

I will be implementing some bot mitigation techniques. This will be the first of changes along with different reader. From the brainstorms, I've learned a few things that may assist in curbing bots, in addition what I thought of implementing. Some of it will be a bit more of an enterprise solution, some of it will be custom solutions, and some of it will just be basic changes that make crawling more of a hassle.

 

These will be added over a long period of time. Almost all of these are machine targeting machine. And you won't even see it.

 

Changes to Reader

To assist with the bot mitigation and reduction of resource usage, the reader will be changed a bit. It will look mostly the same, some might not even notice. But it is coded in a way that'll help me track the abuses more as well as making it lighter weight. So, the users should be able to hit next page faster too. Yay! This is already coded and being tested.

 

Unfortunately, this comes with new URL scheme. Old URLs will be shut down once new is fully operational and without issues.

 

Also, it will now require javascript & cookies to function. So... to the <1% that has it off. Sorry.

 

Membership Requirement

This has been a source of lot of controversy, but I do still find it almost certain it will happen. Yes, some users will leave. Ad revenue may go down, but so will operating costs. They always move together. I foresee no financial concern from this change. On top of that, kenshin has offered more help that's acting as a safety net in terms of cost issues. And if that means I get to have more horsepower to serve the members that care, I see no real concern. I believe what made Batoto into the giant that we have grown into are the contributors. And by contributors, I mean from people who's uploaded thousands of chapters and to people who's made a single comment about their thoughts. Contents are what makes this site, and they already are members.

 

There are no plans to close membership nor anything like a paywall. You can register anytime you want, and become part of this community.

 

Membership requirement will not be a sudden switch. It will be eased in. And what requires membership will slowly grow. The slow pace will give me the numbers to confirm/break my expectations. And that is from where I'll ultimately draw the final choice, not from guesses or what might be. I can't see the future, nor can anyone else from thin air.

 

Uploader's Private Direct Link

This is one change of direction I realized may be helpful after reading the feedbacks in various places. The uploader, optionally can generate a second URL to the chapter. It will not be mentioned anywhere on this site, but people coming to that URL from the source site won't need membership to view. That way, we can still provide a reliable way for uploaders to share without affecting the anti-crawl of our system.

 

I will likely add this sometime while membership requirement is being eased in. Hopefully in early stage.

 

Why are Changes Being Made Now?

Quite a lot seemed confused about this. I do apologize for lack of clarity initially. I've said before, but complete prevention of copying Batoto isn't the goal. It's just not possible. I know that some of them have humans to copy from Batoto in rare occasions that I troll them with fake contents. Goal is to reduce it.

 

Over the years of maintaining Batoto, I didn't do it alone. I have hired other professionals in the past occasionally to help. And their first reaction to my server is always that I'm getting DDoS'ed. If that were true, this is the longest DDoS in history of the internet. (They do eventually know that it's not) We just get crawled that much. We have to get it down to a reasonable level where it's manageable. This, is a viable goal.

 

It's not that Batoto's going to die tomorrow without making the change. Or a month or etc. It's an ever growing problem and has to be tackled sometime. Doing nothing is to just shut down.

 

After

Hopefully focus can be sent back to bunch of stuff that's like 80% done but never finalized... Like follows overhaul. Flattr/changetip that was started like years ago... So much in queue.

 

Self-Published Titles

None of this applies to self published titles. Except automatic bot mitigation.



#2
Grumpy

Grumpy

    RawR

  • Administrators
  • 4,078 posts
  • LocationHere of course!

New reader is now live. Feel free to report bugs in support...  :ph34r:

 

Old reader URLs are still functioning but will be taken down later (roughly a day). I imagine the change will require lots of update for everyone and does cause quite a bit of inconvenience. But crawlers will have saved them and would be able to continue to use past data. I am keeping it up for some time to reduce the inconvenience to the ones who have uploaded, especially recently.

 

I will be monitoring performance to see if everything's nice and smooth. If not, switch back.

 

 

 

Minor point:

With coming holiday seasons, seems like ad networks have a lot of new stuff, which also means lot of bad ads. :( You can report them here which I then forward to networks to resolve. I've also added a few things to cripple the ads a bit in case you do get a bad ad. Should help a little...



#3
Grumpy

Grumpy

    RawR

  • Administrators
  • 4,078 posts
  • LocationHere of course!

So far...

 

  • Seems like we had a bit of troubles with bugs. Resolved a good portion in effect, but many still at large. Actually had to turn off a few checks. Not sure why people are triggering them.
    Please report if you see any issues in the support forum. If you find yourself having to login for the reader, that IS a bug.
  • Server load is down ~30%. So, that's nice...
  • Right now focused on getting reader running and less on security. As members only thing hasn't even been applied yet.


#4
Grumpy

Grumpy

    RawR

  • Administrators
  • 4,078 posts
  • LocationHere of course!

Some more minor updates:

  • Added support back for Opera 12 for you people who has issues letting go.
  • Animation on the reader is removed so that it's faster and better support slower devices.
  • Reader will now cache pages you visited (until you hit refresh or stop reading). So, revisiting pages you visited will be faster.
  • Old URLs are now killed and no longer functioning.


#5
Grumpy

Grumpy

    RawR

  • Administrators
  • 4,078 posts
  • LocationHere of course!

STATS!

The changes has taken place on October 23rd and has now been a week. And this is how the site stats has changed so far:

 

  • 20,000 registrations have been made since the announcement with peak of 3639 registration in 1 day on 25th. And about 1800 registrations per day and continuing as of now.
  • Despite not actually implementing any serious security changes, we're still hovering at about 1/3 less load on our servers simply from change in how our site works. So, if the load avg per server (lower the better) was 3-4 before, it is now 2-2.5.
    Side note: About 5-6 load for us is when all people will start to feel the slow down of the site. So, if the avg is 3-4, peaks often reach into 5s.
  • Loss of users: The most feared aspect expected by all. As of first 2 days, we have had no visible loss beyond daily flux. As of now, we seem to have roughly 10% less people than usual and holding steady. Last 3 days have been mostly a flat line.
  • Sustainability of site: Is more stable than before as operation costs relatively dropped more. Which was the primary objective of the change.

 

Reader URLs (once you have them) are still public and have no membership requirement. I'm pondering if further protection is even required as of now due to slower rebound of bots than expected. But will see and change accordingly. Gonna get me some more stats!



#6
Grumpy

Grumpy

    RawR

  • Administrators
  • 4,078 posts
  • LocationHere of course!

I've found some bugs with the reader and have been fixed. Like sometimes errors were being cached, and thus serving others bad errors.

 

I've also banned a few hundred IPs that has made over 10,000 triggers to my bot tests... If they wanna appeal, hope they have a good reason.