AssemblerGames Backups

  • Thread starter Thread starter 1
  • Start date Start date
Well-known member
Registered
Joined
May 31, 2019
Messages
398
Reaction score
320
Points
63
Haha yeah, I remember there not being much. Mostly drama threads/really old posts. A user has copied all the content so we have it just incase :D
Good to know it was possible to access the data just in case there was something worthwhile.
 
  • Love
Reactions: 1
Member
Registered
Joined
Oct 15, 2018
Messages
17
Reaction score
35
Points
13
As luck would have it, I already wrote a crawler last year for AG that will convert the public posts/threads/forums into a MySQL database. I'm running it now and will see about posting the dump once it finishes.

3IckoNv.png
 
1

1

Guest
As luck would have it, I already wrote a crawler last year for AG that will convert the public posts/threads/forums into a MySQL database. I'm running it now and will see about posting the dump once it finishes.

3IckoNv.png

Having it hosted in a MySQL database would be really handy, I'm hoping that vBulletin3 database I sent is enough :)

EDIT: Running Alpha's script too so we'll have three current post backups.
 
Last edited by a moderator:
Well-known member
Registered
Joined
May 30, 2019
Messages
57
Reaction score
123
Points
33
Any thoughts about loading the content back in to a new forum? I'd thought about the scraping idea, but hadn't bothered. If you've already done it, what about going a step further? If we have a database of all the threads, posts, account names, avatars, etc, it should be possible to write a script that populates a new forum database with that content. You could create legacy accounts like "AG_Nemesis" to "own" the old posts, and blast the content into a new forum.
 
  • Like
Reactions: 1
1

1

Guest
Any thoughts about loading the content back in to a new forum? I'd thought about the scraping idea, but hadn't bothered. If you've already done it, what about going a step further? If we have a database of all the threads, posts, account names, avatars, etc, it should be possible to write a script that populates a new forum database with that content. You could create legacy accounts like "AG_Nemesis" to "own" the old posts, and blast the content into a new forum.

While this would be a good idea much of the older content isn't much value but alot is at the same time. Currently with Alpha's script we're making a database of posts/threads/usernames and the intention is to make a static clone of AG on another domain. This would be controlled via MySQL if there was ever a need to merge here but it wouldn't be able to be logged into. Akira is helping there with that and I'll be doing the html/css side to make it look like AG minus images etc although I may manually add those not sure yet.
 
AG Refugee Retro Brony
Registered
Joined
Jun 1, 2019
Messages
59
Reaction score
46
Points
18
I might can help with doing more archiving if needed. I have terabytes of storage space and gigabit internet that will give me full speed for both DL and upload. PM me if that would be useful to archive anything else. Only hidden stuff I can access on AG is the 0th bit.
 
Well-known member
Registered
Joined
May 30, 2019
Messages
57
Reaction score
123
Points
33
While this would be a good idea much of the older content isn't much value but alot is at the same time. Currently with Alpha's script we're making a database of posts/threads/usernames and the intention is to make a static clone of AG on another domain. This would be controlled via MySQL if there was ever a need to merge here but it wouldn't be able to be logged into. Akira is helping there with that and I'll be doing the html/css side to make it look like AG minus images etc although I may manually add those not sure yet.
Be aware that the backup I did in December last year is a perfect clone of the visual content of the site in a 100% standalone fashion (IE, you can load it in your local browser offline and it'll appear identical to the real site). I've just done a full mirror of the site again using the same process, which I'll upload soon. If you just want to make a static site mirror, that clone will function well in that regard. I don't have access to the 0th bit section to incorporate that into the mirror, but if someone is willing to lend me access to their assemblergames account who has access to that section, I could incorporate that too. I'll be publishing instructions and tools to help people backup their PMs in the next little while too. I had hundreds of PMs I wanted to save, so I've rigged up a more limited mirror that'll function to capture just that content fully intact (IE, images, attachments, and so on).
 
AG Refugee Retro Brony
Registered
Joined
Jun 1, 2019
Messages
59
Reaction score
46
Points
18
Be aware that the backup I did in December last year is a perfect clone of the visual content of the site in a 100% standalone fashion (IE, you can load it in your local browser offline and it'll appear identical to the real site). I've just done a full mirror of the site again using the same process, which I'll upload soon. If you just want to make a static site mirror, that clone will function well in that regard. I don't have access to the 0th bit section to incorporate that into the mirror, but if someone is willing to lend me access to their assemblergames account who has access to that section, I could incorporate that too. I'll be publishing instructions and tools to help people backup their PMs in the next little while too. I had hundreds of PMs I wanted to save, so I've rigged up a more limited mirror that'll function to capture just that content fully intact (IE, images, attachments, and so on).
Check PM.

EDIT: I can get a domain in 2 days when I have money. ASSEMblerarchives is available in every way if we wanna use that to host the archived site under. Will cost me $12 USD a year for the domain if I use my Google domains to get it.
 
Last edited:
New member
Joined
Jun 2, 2019
Messages
4
Reaction score
9
Points
3
That's okay. There wasn't that much worth bothering with in that section either. Funny thing, when we quit being mods we were kicked out of the 0 bit section.

Nothing changed after you guys were locked out, in terms of new content.
 
AG Refugee Retro Brony
Registered
Joined
Jun 1, 2019
Messages
59
Reaction score
46
Points
18
Yeah. There has been no activity in there until recently. I was the last to post in the roll call thread. That was several years back. Makes me wonder if I'm one of the last to be given the role.
 
Well-known member
Registered
Joined
May 30, 2019
Messages
83
Reaction score
136
Points
33
Check PM.

EDIT: I can get a domain in 2 days when I have money. ASSEMblerarchives is available in every way if we wanna use that to host the archived site under. Will cost me $12 USD a year for the domain if I use my Google domains to get it.

We have a suitable domain already, which will host the mirror.
 
AG Refugee Retro Brony
Registered
Joined
Jun 1, 2019
Messages
59
Reaction score
46
Points
18
OK. That's good. You all here went the whole 10 yards with this.
 
  • Like
Reactions: 1
Member
Registered
Joined
May 31, 2019
Messages
20
Reaction score
2
Points
3
Really nice work on the script Alpha, I'm interested in what the end-result be like here.
 
1

1

Guest


Progress on the @Alpha crawler rip. About 40k threads, 700k threads left and about 17k members before it finishes. Akira's already working on the front end to display the data in a static version of AG on a domain we've purchased in advanced. Only thing to add to that is a search system which another members working on.
 
Well-known member
Registered
Joined
May 30, 2019
Messages
57
Reaction score
123
Points
33
You should be aware, I'm about to drop my backup of the site, which is more complete and ready for use than the wget mirror you've posted. Backup is done, cleanup is done, archive is compressed, just needs uploading, which I'll get done in the next couple of hours.
 
1

1

Guest
You should be aware, I'm about to drop my backup of the site, which is more complete and ready for use than the wget mirror you've posted. Backup is done, cleanup is done, archive is compressed, just needs uploading, which I'll get done in the next couple of hours.

We've got an HTTRACK Mirror running too, as mentioned in first post :) along with a custom built crawler that's auto populating a MySQL database, check the post in news once I'm done for more info thanks, yours wll be compared just to be sure ours is good before sticking on archive.org.
 
Well-known member
Registered
Joined
May 30, 2019
Messages
57
Reaction score
123
Points
33
Alrighty, just don't underestimate how pedantic I am ;). This is no ordinary build of httrack, and I spent over a month perfecting the ripping process last year. For example, I made sure to rip all the forum index pages in a 5 minute window, then checked back on the site to ensure no topics fell through the cracks by a new thread being added to page 1 for example, before I'd retrieved page 64. I made sure every single page of every topic got scanned, and got scanned only once, and I wrote tools to verify the results. I renamed attachments, ripped all important external content (such as included css and embedded images), and spent hours browsing the backup with no internet connection to make sure it looked identical to the real site. I made sure you could view poll results, use the dynamic "arrows" to build valid page navigation links when browsing long topics/forums, and I even made sure the favicon is properly referenced on every page so that it appears for the complete site, even if you open it locally or don't host it at a root domain level. I had to make lots of fixes to the engine too, in order to make sure topics with unicode characters in the URL like this one got mirrored properly, as well as fix a bunch of other issues. The list of rules is so long I couldn't pass it as command line arguments normally, as it blew the character limit, I had to launch the process through code to get the arguments in. Basically, I obsessed over it and spent far too long making sure it was just right, and I've been through that process again now, with improvements over last time. I'd strongly recommend you try out my rip (with your internet disconnected!) before you consider hosting anything. I think you'll find there's not much else to do than drop the files on a webhost and walk away.
 
Top