-
ServLib Manual Scrapper
01/20/2022 at 08:04 • 0 commentsI made a scrapper script in python to download full-res manuals from Serv L1b à la Sci-Hub and turn them into PDF
Access the manuals from ServLib.
This is for educational purposes on how to find and archive data. Who the heck would need the manual for a KX-FT33LA Thermal Printing Fax MachineThe code is quite explained, but you can direct any questions by submitting an issue or messaging me at @AriZoneVibes :
You can find the file in my GitHub
You will need to have python installed on your computer.
Once installed, run the following commands to install the libraries needed to run the script
requests - Used to make HTTP requests. Will ask the website for the image.
pip install requests
pyFPDF - Library for PDF document generation. We will use it to take the downloaded images and place them into a pdf
pip install fpdf
Obtaining the link
As an example I'll be using the manual for KX-FT33LA Fax Machine, browsing through Firefox
Access the manual, right-click over the page/image and click "Inspect Element".
On the developer console, go to the upper menus and press "Network". (It might ask you to reload)
Browse for the .PNG files on the list that shows up. Hover over it and should show a thumbnail. Confirm it's from the manual you are browsing and click.
Another view should open. Copy the link after GET and continue, we will now work on the script.
Script
Open
Scrapper.py
file in your favorite text editor. Modify the settings according to the pages you need and if it should make a PDF file from all the pages.#Settings Url = "https://servlib.com/disk/panasonic/fax/kx-ft33la/html/km79804202c3-1.png" #Url you got on the previous step, should look like: https://servlib.com/disk/panasonic/fax/kx-ft33la/html/km79804202c3-1.png StartPage = 1 #First page to export EndPage = 127 #Last page to export MakePdf = True #If will export the final file as PDF
The last page value is as indexed on the website, not on the manual itself.
Save the file and open a terminal in the location where you want your files stored. Lastly, run the script. Will display the progress as the images are downloaded.
python3 Scrapper.py
If you want to learn more about how the script works you can find information on the comments inside the
Scrapper.py
file -
ESP32-Cam Arduino Flashing Quirks
07/14/2021 at 19:42 • 0 commentsI got some ESP32 cam boards but I forgot about getting the serial communication board. As a replacement I used my arduino to get the serial communication. The firmware flashing worked using espwebtools but not using esptools and platformio/arduino ide
After a day of testing I figured all the details I needed to change. This is mostly for reference for when I get to flash them again. But may also be helpful for others, specially the platformio settings addition since the tuning of this is never mentioned on online tutorials about this
Platformio:
Add this setting into the platformio.ini file or from the project home.
upload_speed = 115200 monitor_speed = 115200
Cable Connections:
Use 3.3v when flashing
Arduino: GND-Reset
ESP32: GND-IO0Arduino ESP32 GND GND 3.3v 3.3v TX UDT RX UnR Press Reset button on ESP32 board when the serial monitor says "Connecting.....-----....."