Announcement

After 5 years serving the vintage Apple enthusiast community, ThinkClassic has been marked for closure and is now in caretaker mode. Please see this thread for further information. Please direct any questions, comments and enquiries about the website, management and ownership to this thread.

You are not logged in.

#1 2014-07-13 19:52:12

bbraun
Member
Registered: 2014-05-29
Posts: 1,064
Website

ThinkClassic native app

Rather than continuing to hijack the thread in the Front Bar, this thread is where I'll post updates to the screen scraping native app.

As background, this is a System 7 app to view and post to thinkclassic.org.
The current requirements are: System 7.0.1 or later, and MacTCP or OpenTransport.

Changes in 0.2:
- I implemented some basic unescaping of the forum & topic names, and post text, consisting mainly of understanding "&" and "'", which seem to be the most commonly used escaped characters so far.
- Post text is also de-HTML'd, which is a little easier to read. 
- It'll also try to login on launch, rather than needing to explicitly login from the File->Login menu option.
- I'm scraping the username of the poster and including that, so you at least know whose post you're reading.
- It should handle posts with multiple pages better.

tc02.png

Source and app: thinkclassic0.2.cpt.hqx

Offline

#2 2014-07-13 20:29:19

uniserver
Member
From: Sf, Mi
Registered: 2014-05-15
Posts: 956
Website

Re: ThinkClassic native app

wow you could make apps like this for other sites as well?     - too cool.


#I Re-Cap √Mac √NeTX √Amiga Boards - A/B - PSU# (MacCaps.com)  Modern SCSI HD's - For Old Macs - Pre Cfg'd - 10k RPM! 73gb!! $50 + free shipping  -- Mac 128K Re-Ram kits (16 Chips) $35 + shipping, Floppy Issues?-> Bourns Filter Solution 128k - SE/30, $16 + shipping

Offline

#3 2014-07-13 23:38:24

bbraun
Member
Registered: 2014-05-29
Posts: 1,064
Website

Re: ThinkClassic native app

It is possible, as I like to say with software anything is possible, given enough time.  But that's the real trick, there's never enough time.
Since this isn't really parsing the html, just kind of doing some substring matching, it is prone to breaking anytime there's changes to the site.  Different sites have different capabilities and challenges though.  The larger sites have web programming interfaces (like REST, XML RPC, whatever the new hawtness is), which make things a bit easier.  On the other hand, most places require SSL, which makes things a bit harder.  I tend to shy away from doing anything where I'd have to spend much time on the authentication system (like most of the major social networking sites), since I do authentication systems for my day job, and it becomes a little to much like 'work'.  This is all about screwing around, having fun, and hopefully learning something in the process.  smile

This is an idea I've been tossing around for a while.  I might have mentioned doing one for 68kmla at some time in the past, and certainly considered it.  With mac68k.info there wasn't much point since it has a fully functional NNTP interface.  TC was just the right place, right setup, at the right time.

Offline

#4 2014-07-14 01:09:31

ClassicHasClass
Member
From: Electron Alley
Registered: 2014-05-26
Posts: 1,118
Website

Re: ThinkClassic native app

Okay, I had to try it. Installed and compiled it locally with my own copy of CodeWarrior Gold. Works great. Nice job!


Machine room (updated for 2019!): http://www.floodgap.com/etc/machines.html

Offline

#5 2014-07-14 02:18:44

bbraun
Member
Registered: 2014-05-29
Posts: 1,064
Website

Re: ThinkClassic native app

Awesome!  It's always good to know it works for someone other than myself.

Offline

#6 2014-07-14 03:50:58

uniserver
Member
From: Sf, Mi
Registered: 2014-05-15
Posts: 956
Website

Re: ThinkClassic native app

Do you think maybe you could just post the binary?


#I Re-Cap √Mac √NeTX √Amiga Boards - A/B - PSU# (MacCaps.com)  Modern SCSI HD's - For Old Macs - Pre Cfg'd - 10k RPM! 73gb!! $50 + free shipping  -- Mac 128K Re-Ram kits (16 Chips) $35 + shipping, Floppy Issues?-> Bourns Filter Solution 128k - SE/30, $16 + shipping

Offline

#7 2014-07-14 03:52:52

bbraun
Member
Registered: 2014-05-29
Posts: 1,064
Website

Re: ThinkClassic native app

The archive has both the binary and the source.  Do you mean an archive without the source?

Offline

#8 2014-07-14 05:46:55

uniserver
Member
From: Sf, Mi
Registered: 2014-05-15
Posts: 956
Website

Re: ThinkClassic native app

ok from classic has class's response i assumed that maybe you just included the source and it had to be compiled HAHA...  sorry.


#I Re-Cap √Mac √NeTX √Amiga Boards - A/B - PSU# (MacCaps.com)  Modern SCSI HD's - For Old Macs - Pre Cfg'd - 10k RPM! 73gb!! $50 + free shipping  -- Mac 128K Re-Ram kits (16 Chips) $35 + shipping, Floppy Issues?-> Bourns Filter Solution 128k - SE/30, $16 + shipping

Offline

#9 2014-07-14 08:04:26

iMic
Administrator
From: Adelaide, Australia
Registered: 2014-05-12
Posts: 948
Website

Re: ThinkClassic native app

Impressive! Will have to try it myself some time.

We do have some site changes in the pipeline, but I'm fairly confident that the HTML structure will remain the same. There's a good chance that this app won't require any changes to make it work after the update, but at the very most it'll be one or two changed paths. Future releases of FluxBB may change the HTML structure, but we should know if that's to be the case ahead of schedule.

Our current version (1.5.6) will likely be updated to 1.5.7 when it's released, unless they choose to fork it to the 1.6.x branch. Even if that's the case, the HTML should remain fairly consistent between releases. The only exception I can see is when they move from the 1.x branch to the 2.x branch since that's a complete rewrite, but I don't think that will happen for some time.


Resident Professor of Alternative Methodology
Faculty of Electronic Restorations & Modifications - "It works, let's fix it!"

Offline

#10 2014-07-14 16:36:37

markyb
Member
From: Aurora, OH (330)
Registered: 2014-05-16
Posts: 185
Website

Re: ThinkClassic native app

Now that's awesome. Always something cool coming from bbraun. big_smile


http://markyb86.weebly.com for some packs of old macintosh wallpapers, desktop patterns, windows wallpapers, sound files, etc.

Offline

#11 2014-07-14 16:54:44

cc333
Member
From: North S.F. Bay Area, CA
Registered: 2014-05-23
Posts: 600

Re: ThinkClassic native app

I know! This is really neat!!

None of my classic Macs are internet enabled currently, but I will certainly give this a try at some point (maybe in an emulator? What would be the point of that, though?)

c

Last edited by cc333 (2014-07-14 16:54:54)


Main Macs: Early '09 Mac Pro, Mid '12 MacBook Pro 13"
Secondary Macs: Early '08 Mac Pro, Mid '12 MacBook Pro 15"
Playthings: Mac SE/30, 3.0 GHz Mavericks-based HackServe, Many others....
Desired: Lisa, Kanga PowerBook G3, Apple IIc, Apple II, Spare parts, etc.

Offline

#12 2014-07-16 18:57:06

bbraun
Member
Registered: 2014-05-29
Posts: 1,064
Website

Re: ThinkClassic native app

Neat.  This is being posted from a pb100 with System 6 and the above binary.

Offline

#13 2014-07-17 04:37:56

ClassicHasClass
Member
From: Electron Alley
Registered: 2014-05-26
Posts: 1,118
Website

Re: ThinkClassic native app

What are you using for connectivity? SCSI? MacIP?

I think you just won, unless someone gets this on a Mac Plus.


Machine room (updated for 2019!): http://www.floodgap.com/etc/machines.html

Offline

#14 2014-07-17 05:25:15

bbraun
Member
Registered: 2014-05-29
Posts: 1,064
Website

Re: ThinkClassic native app

Yeah, I'm using a scsi ethernet adapter.  I just tried putting a modem from my pb180 into the pb100, to use the line simulator/rpi setup I used for the datarover, but something ain't right, so ethernet it is for the moment.

I was mainly trying to see what I'd have to do to make it work on System 6, but it worked just fine without any changes.

Last edited by bbraun (2014-07-17 05:28:11)

Offline

#15 2014-07-17 07:39:30

uniserver
Member
From: Sf, Mi
Registered: 2014-05-15
Posts: 956
Website

Re: ThinkClassic native app

so pretty much no matter what ... BBraun wins this one smile

http://www.thinkclassic.org/viewtopic.php?id=15


so the question is : what does he win?

i have a re-capped LC-III too send him, if someone wants to chip in and pay his shipping :-)

Last edited by uniserver (2014-07-17 07:42:15)


#I Re-Cap √Mac √NeTX √Amiga Boards - A/B - PSU# (MacCaps.com)  Modern SCSI HD's - For Old Macs - Pre Cfg'd - 10k RPM! 73gb!! $50 + free shipping  -- Mac 128K Re-Ram kits (16 Chips) $35 + shipping, Floppy Issues?-> Bourns Filter Solution 128k - SE/30, $16 + shipping

Offline

#16 2014-07-18 15:11:43

ClassicHasClass
Member
From: Electron Alley
Registered: 2014-05-26
Posts: 1,118
Website

Re: ThinkClassic native app

I updated CodeWarrior on the Q800 to Pro 2 and it needs a couple tweaks to compile. In particular, Gestalt.Equ.h should now be Gestalt.h and the enums for FindControl() (like inUpButton, etc.), need to be the later k* equivalents. And of course, the new names for the MSL and ANSI libraries.

I was going to update it to Pro 5 since I have it also but I forgot the IDE doesn't run on 68K. hmm


Machine room (updated for 2019!): http://www.floodgap.com/etc/machines.html

Offline

#17 2014-08-05 01:06:46

Edgecrusherr
Member
Registered: 2014-07-30
Posts: 8

Re: ThinkClassic native app

Nice job! If someone could make something like this for Facebook, I think that would be the current most popular application for classic Mac OS lol.

Offline

#18 2014-10-26 18:29:10

bbraun
Member
Registered: 2014-05-29
Posts: 1,064
Website

Re: ThinkClassic native app

I've updated the native app so it continues working.
The changes are basically to just add the User-Agent string so it can be whitelisted from the sslification efforts, and some minor tweaks to keep working with site layout.
thinkclassic0.3.cpt.hqx

Offline

#19 2014-10-26 21:26:03

ClassicHasClass
Member
From: Electron Alley
Registered: 2014-05-26
Posts: 1,118
Website

Re: ThinkClassic native app

I love native clients.


Machine room (updated for 2019!): http://www.floodgap.com/etc/machines.html

Offline

Board footer

About ThinkClassic

ThinkClassic specialises in the use, maintenance, repair, restoration and modification of vintage computers and peripherals.