/operate/ - Endchan Operations

Let us know what's up

Posting mode: Reply

Drawing x size canvas

Remember to follow the rules

Max file size: 350.00 MB

Max files: 5

Max message length: 4096

Manage Board | Moderate Thread

Return | Catalog | Bottom

Expand All Images

(238.57 KB 960x1280 image.jpg)
If I wrote 8chan migration for InfinityNow Anonymous 01/11/2017 (Wed) 02:10:40 [Preview] No. 5549
To OdiliTime and SnakeDude:

Would you be open to the idea of allowing endchan board owners to rip and migrate 8chan threads and their replies to their endchan board?

The purpose of this would be to allow less active 8chan board communities that don't want to start fresh to migrate here as seamlessly as possible.

It's an ambitious project that I'd like to try writing for your website, but I don't want to spend weeks on this only to be told that you have zero interest in allowing something like this to be implemented on your website in the first place, even if the code meets your performance and security expectations and very cleanly migrates data to the MongoDB. I need to know if you'd be open to this if I ever manage to get it at a finished, thoroughly tested state.

I'm leaving the technical details out for now in this OP to be keep this post short and to the point.

Anonymous 01/11/2017 (Wed) 02:16:22 [Preview] No. 5550 del
Btw the ripping program that I would write would be separate from endchan (downloadable from gitgud.io) if that wasn't obvious for plausible deniability reasons. There would be a migration extension written for InfinityNow. The idea is to be able to be able to upload the ripped board package to account.js. I've done some rough calculations: a 25-page board like /monster/ on 8chan would be 6GB large. But this project isn't for active boards like /monster/ anyway.

Anonymous 01/11/2017 (Wed) 02:17:21 [Preview] No. 5551 del
do you have the script or on the process of writing one?

Anonymous 01/11/2017 (Wed) 02:25:47 [Preview] No. 5552 del
The board ripper is for 8chan board owners by pulling data from sys.8ch.net/mod.php?/[board]/

Anonymous 01/11/2017 (Wed) 05:30:08 [Preview] No. 5553 del

how does the exported data look like?

would it make sense to write script that manually migrates post by simple http(s) requests to endchan's board one by one?

Anonymous 01/11/2017 (Wed) 05:32:32 [Preview] No. 5554 del
just my 2c, but i doubt they would want to make this an official feature of endchan and place it on account.js moderator page for everyone to use due to feature bloat and liability reasons. Frankly your code would be for a very special case and they seem to be keen on keeping the codebase as lean as possible.

on the other hand, i do think it would be very nice of them if they were to allow this as an "unofficial" option where 8chan board owners can privately request the endchan staff to migrate their 8chan board's threads to the corresponding endchan board by sending them the ripped data (name, date, post #, files, etc.)

so i do think it would be a neat option, and you should still write your board ripping program that rips and formats the relevant data from the static html mod pages and then write the migration script so that data can be migrated to infinitynow/lynxchan. but don't expect this kind of thing to ever become an official feature, it just seems to me a very special case that would have to be something endchan staff does manually, if they're even willing. also, i don't know how much 8chan's codebase is changing, but it could potentially be a nightmare to maintain this ripper+migrator since 8chan is not open source and you don't always know what changes behind the scenes.

odilitime Board owner 01/11/2017 (Wed) 15:39:02 [Preview] No. 5555 del
Well look at these, probably could be converted to using the 4chan JSON API instead of using mysql to read infinity

Definitely some ethical issues with steal content.

Then the question of does this really solve what you're trying to do. And I would say no. If you don't have a big enough user base or the will to set up shop new, you're not likely just to move cause of the content has been copied.

but I'm open to the idea and discussion about it.

Yea, I think it would have to be a supervised process like claims. You'd send a request, we'd process it and then that's that.

I maintain 8archive, the 4chan API is superstable, it's not a big deal to maintain at all.

Balrog Board volunteer 01/11/2017 (Wed) 21:07:59 [Preview] No. 5557 del
For the record, in the earliest phase when the goal was to onboard the non-shit people from 8chan as quickly and smoothly as possible before Next dropped and scattered everyone, I had considered some kind of scraping setup to help content-dump-centric boards (e.g. /pdf/) get back up to speed. That sort of fell by the wayside when the plan to give 8chan BOs first dibs on their boards on endchan (again, to try and skip the period of anarchy post-Next) was met with a resounding "fuck you" and the exodus died.

While I would normally be at least a bit concerned about the ethics of scraping boards wholesale, in my opinion 8chan has decayed to the point where the staff is beyond any consideration of ethics or mercy. Fuck Watkins and everything he touches, any questions of ethics lie with the community of the board to be scraped.

The real issue with scraping (and transferring 8chan boards to endchan intact as a prerequisite) would be global staff picking winners and losers and giving 8chan communities a head start. Again, I tried that (the only boards that actually got to take advantage were /tg/ and /monster/, since I went to them first to provide a low-cancer vanguard; they've still kept endchan as their primary bunker) and the immediate reaction over here was "fuck 8gaggers, let them come here on their own and rise or fall on their own merits." So it would really be up to the community here to decide if it's worth it. If it's decided to be a good idea it would be like the early plan.
>8chan board owner verifies with a capcode post that they do own /foo/
>we run migration script on 8chan's /foo/ pointing to endchan's /foo/
>8chan board owner has ownership of endchan /foo/ transferred to their endchan moderator account

odilitime 01/12/2017 (Thu) 10:27:29 [Preview] No. 5560 del
After some thought, if the community created the content, it's likely their content. Unless 8chan has some really shitty legal rules.

Top | Return | Catalog | Post a reply