36 replies to this topic

[fuzzyk]

Well in hindsight a slightly different architecture might have been better, yes. However, the main source of problems is me downloading the cover images. They simply take a lot of space, especially because some are insanely large. I will simply move this process so that its results aren’t kept in memory but written to database right away. A new version should be available within 2 hours.

 

The technical details are like this: A dictionary needs a reasonably-sized chunk of contiguous memory to store its hash table. It’s really not that much. However, because of all the images floating around, memory fragmentation means that while in total there’s enough free memory, there isn’t enough contiguous memory. If the application is 64 bit, the much larger memory space means that this isn’t an issue anymore and you can better utilize your physical memory even with somewhat sloppy programming.

 

Of course, all this takes place on the heap, not the stack.

Ok so having tried running the grabber 3 times i have gotten the same issue all 3 times. While the program is running through the various groups warning pops up that says "CefSharp.BrowserSubProcessor has stopped working". It seems to happen at the same point every time at 236 out of 263 groups scanned. The scan did finish, but i haven't had the time to check to see if the file made works correctly. Any ideas? Is this a potentially serious issue? Is there any way i can provide more info to help fix this issue?

 

https://imgur.com/a/YtgUZ

Sounds like you’re encountering a bug in CefSharp. Nothing really I can do about that, sorry. I will upgrade to latest the CefSharp version, maybe that’s enough.

 

In the meantime, make sure all your drivers are up to date. Because it uses GPU acceleration, your graphics driver is especially important.

[fuzzyk]

Well, clearly I’m terrible at managing my time. I’m currently testing my changes, looks good so far. Memory usage after fetching 279 series and 100 (of 305) groups is at ~67 MiB.

 

Images are not kept in memory anymore. As such, the tool should now be able to handle virtually unlimited amounts of series and stuff.

The new version is now available.

 

Unfortunately, I was not able to upgrade the CEF runtime to a newer version due to version constraints.

[Forever Young]

Mine ran all the way, at the end a name was required to save the file. I wrote the name and then it gave an error and didn't save anything.

I couldn't take a screenshot but I'm running it for the second time. I hope it can save the data this time. 

Anyone getting this error? I mean, saving error?

 

edit: I had 400 sth manga to save. not so much I guess.

Edited by Forever Young, 19 January 2018 - 06:14 PM.

[fuzzyk]

Sorry to hear it's not working for you! However, without the error message, I don't know what's wrong.

[Forever Young]

Thanks for the answer.
This time I got another error
 

Spoiler

and it stopped working. I unfollowed this manga and went for a third round.

[fuzzyk]

Thanks for the info. I will take a look at this series' page and try to figure it out. It does appear to have a cover image after all, so maybe my script is looking at the wrong place.

[Equisdesu]

Whenever it gets to the end, after naming the file I get this error:

 

Spoiler

 (Attached screenshot)

"System.DllNotFoundException: Can't load file DLL 'SQLite.interop.dll': Can't find specified module. (HRESULT exception: 0x8007007E)" 

 

Honestly I've not Idea what I'm doing, I'm just really interested in keeping the info of the groups for the mangas I'm following here. I'd appreciate any help!

 

Also for what it's worth, I'm using the 1.5 from the releases page and I'm running Windows10 64bits. (also I'm only following 96 mangas, so I guess that's not much)

Attached Files

•  BatotoGrabber_2018-01-19_15-59-08.png   12.93KB   0 downloads

[fuzzyk]

You’re right, I made a mistake while packing version 1.5. Sorry! I’ll have the fixed version uploaded in ~5 minutes.

Edited by fuzzyk, 19 January 2018 - 07:26 PM.

[bargiddle]

using 1.5 im getting the following error when attempting to save after scanning everything

System.DllNotFoundException: Unable to load DLL 'SQLite.Interop.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)

   at System.Data.SQLite.UnsafeNativeMethods.sqlite3_config_none(SQLiteConfigOpsEnum op)

   at System.Data.SQLite.SQLite3.StaticIsInitialized()

   at System.Data.SQLite.SQLiteLog.Initialize()

   at System.Data.SQLite.SQLiteConnection..ctor(String connectionString, Boolean parseViaFramework)

   at System.Data.SQLite.SQLiteConnection..ctor(String connectionString)

   at BatotoGrabber.MainForm.<Grab>d__5.MoveNext()

[fuzzyk]

Yes, this error is now fixed. However, you must download version 1.5 again from the GitHub releases page.

and it stopped working. I unfollowed this manga and went for a third round.

Unfortunately, I wasn’t able to find anything wrong with this series’ page. I have improved upon the robustness of this script in version 1.6. If you encounter this error again, I recommend you try the new version!

[Forever Young]

um... maybe it's kinda my system's fault but this time i got this error. sorry to bother you again. i hope it helps.
 

Spoiler

 
like Equisdesu, i'm only interested in the group's info. maybe the sone of batoto image servers' shutting down is affecting the app.

edit: ok i'm downloading the 1.6 version now. thanks

Edited by Forever Young, 19 January 2018 - 07:51 PM.

[bargiddle]

1.6 works great, thanks so much for this

[fuzzyk]

maybe the sone of batoto image servers' shutting down is affecting the app.

No, at that point only the HTML served by Batoto is relevant. Though that also seems somewhat flakely currently. Considering how you’re looking at an empty page and all.

[Forever Young]

hello again.

this time I got no errors and I was able to save and browse the data.

I was a little bit disappointed at first because I thought with this tool we would be able to download the chapters, too. 

Totally my ignorance lol.

 

I took me a bit of time to figure out how to browse the data but it is just normal for a total newbie like me.

 

Thank you so much for all your efforts Fuzzyk.

Edited by Forever Young, 19 January 2018 - 09:47 PM.

[fuzzyk]

No problem. Downloading the chapters is much more complicated task, in regards to both technique as well as storage. Also, I could not openly promote a tool that does that, even in Batoto’s final days.

[Equisdesu]

Just downloaded 1.6 and it worked perfectly, thanks a lot!

[Natureboy]

Thanks for the fix to version 1.6.  I also had the problem with an error on file-save in version 1.5