DazArtificer - Ultimate Scraping Tool

Home Page Forums General Chat DazArtificer - Ultimate Scraping Tool

Viewing 15 posts - 16 through 30 (of 49 total)
  • Author
    Posts
  • #2060843
    Redflag
    Participant
    Rank: Rank 3

    There was a tool on F95... Called
    Daz Content Manager

    It was awesome. It worked in conjunction with Jdownloader, and would scrape the vendor sites, along with the main pirate sites, and make a list of all the new stuff available. A couple of clicks got whatever you wanted in your Daz Library.

    THe Zone here wasn't involved, but since this is the main feeder site to 95% of all others....

    Anyway, the developer lost interest and it kind of died... if your program could add that functionality, it would be worth its weight in gold.

    I believe it is written in Java or something... but the developer posted the code in case anyone else wanted to take over (scroll down on page two for the files)

    #2061283
    Lewd18Ratio
    Participant
    Rank: Rank-2

    Thanks for your feedback.

    It's a really interesting tool.

    But right now I focuse on developing the tool for content uploaders.

    I will still add it to my ToDo because, like you said it's a really helpful functionality, especially the content management.

    #2061284
    Lewd18Ratio
    Participant
    Rank: Rank-2

    I updated my tool to v1.3 and added some new features.
    The new update can be found in the same Main Folder.

    V1.3

    ### Added
    - Vendor information scraping and inclusion in archive naming.
    - Custom naming pattern for compression archives in settings, enabling users to include {TITLE}, {SKU}, {DIM}, {VENDOR} in the file name.
    - Custom DIM format setting, allowing for short (NDIM, CDIM, ODIM) or default (No DIM, Custom DIM, Original DIM) representations in archive names.
    - Folder compression support, allowing users to archive directories in addition to individual files.
    - Thumbnails support for inclusion in the compression archives.
    - Option in settings to enable or disable the compression of thumbnail images.
    - Renaming the promo image to "main_image" in the compression archives for consistency.

    ### Changed
    - Improved scraping logic for descriptions, now supporting HTML format to preserve some original formatting.
    - Enhanced GUI and interactions for a more intuitive user experience.
    - Refactored code organization into modules for better maintainability.
    - Renamed variables in the code for clarity and consistency.
    - Updated settings dialog to accommodate new compression naming, DIM format options, and thumbnail compression settings.

    ### Fixed
    - Fixed an issue where descriptions on Renderotica were sometimes not found.
    - Addressed various stability and error handling improvements to ensure smoother operation.
    - Adjusted the handling of image URLs to remove parameters that could lead to loading smaller or incorrectly formatted images.

    DazArtificer V1.3

    #2061615
    Lewd18Ratio
    Participant
    Rank: Rank-2

    New Update out!

    Newest Version is v1.4

    Changelog:

    v1.3.5

    ### Added
    - Clear button added to GUI for resetting all input fields and selections for a fresh start.
    - Support for multiple files and folders selection for compression, enhancing usability for users who work with numerous assets.

    ### Changed
    - Improved error handling to prevent the upload of directories to MEGA, as only file uploads are supported.
    - Adjustments to the GUI layout to accommodate new features and improve user experience.

    ### Fixed
    - Fixed and refined the GUI elements to ensure a consistent and intuitive user interface across different sections of the application.
    - Minor bug fixes and performance improvements to enhance application stability.

    v1.4

    ### Added
    - Compatible Figures scraping functionality, enabling users to obtain detailed compatibility information directly from product pages across supported websites.
    - Required Products scraping functionality, which identifies and lists products necessary for the use of the scraped item, enhancing the comprehensiveness of the data collection process.
    - Link to Icons8 in the FAQ section to acknowledge the source of icons used in the application, promoting transparency and giving credit to the icon provider.

    ### Changed
    - Modified the GUI to include new fields for Compatible Figures and Required Products, ensuring users have access to this vital information at a glance.
    - Adjustments to the scraping logic to accommodate the extraction of Compatible Figures and Required Products from product pages, reflecting a more holistic approach to data collection.
    - Enhanced the user interface to remove Required Products information from the details section in Renderhub, streamlining the presentation of information and reducing redundancy.
    - Improved the layout and functionality of the GUI to accommodate the addition of new features, ensuring a seamless user experience.

    ### Fixed
    - Addressed an issue where images were sometimes not uploaded due to URL parameters (e.g., "?") affecting image retrieval, particularly in Renderhub. The application now correctly handles such URLs to ensure reliable image uploads.
    - Minor bug fixes and GUI refinements to address user feedback and enhance application stability, ensuring a smoother overall user experience.

    ### Acknowledgments
    - Included a link to Icons8 in the FAQ to acknowledge their contribution of icons used within the application, promoting transparency and giving appropriate credit.

    DazArtificer v1.4

    Download DazArtificer - Click here for the official DazArtificer post.

    #2062941
    Lewd18Ratio
    Participant
    Rank: Rank-2

    New Update to v1.5 is out!

    Changelog:

    v1.5

    ### Added
    - Custom MEGA upload folder option in settings, allowing users to specify a default folder in their MEGA account for uploads. This enhances organization and ease of access to uploaded files.
    - Compatibility Software scraping functionality, enabling users to extract and display software compatibility information from product pages across supported websites. This feature provides valuable insights into the software requirements for the use of the scraped item.
    - Product information HTML file (product_info.html) generation during file compression, offering a detailed overview of the product, including title, SKU, compatible figures, required products, and a description. This addition enriches the content of the compression archives, providing comprehensive product details at a glance.
    - Setting to enable or disable the creation of product_info.html within the compression archives, giving users control over the inclusion of detailed product information in their archives.

    ### Changed
    - The Browser Engine selection is now saved in QSettings, preserving the user's choice across sessions and enhancing the user experience by eliminating the need to reselect the browser engine for each use.
    - Improved handling of SKU and data retrieval on renderosity.com to accommodate unique URLs, ensuring more reliable and consistent data scraping from the site.

    ### Fixed
    - Resolved an issue where the SKU was sometimes not found on renderosity.com due to variations in page structure or content.
    - Addressed a problem where data could not be retrieved from renderosity.com for certain products because of unique URL formats or page structures, enhancing the robustness of the scraping functionality.
    - Minor bug fixes and adjustments to the user interface to improve application stability and usability.

    DazArtificer v1.5

    #2064496
    Lewd18Ratio
    Participant
    Rank: Rank-2

    New update to v1.6 is out!

    Changelog:

    v1.6

    ### Added
    - "Auto-Delete Originals" option for automatically deleting original files and folders after compression.
    - Support for data scraping from cgbytes.com.
    - "Auto-Open Archive" option to automatically open the archive file after compression.
    - "Auto-Open Folder" option to automatically open the folder containing the archive file after compression.
    - Slider option for selecting the compression level from 0 to 9, allowing users to customize the balance between compression efficiency and file size.

    ### Changed
    - Updated scraping logic for compatible software on daz3d.com to exclude items like Daz to Maya Bridge, Daz to Unity Bridge, Daz to Unreal Bridge, etc., focusing on relevant software compatibility information.

    ### Fixed
    - Resolved issues where compatible software information was sometimes not found on renderotica and cgbytes, improving the accuracy and completeness of scraped data.

    #2065007
    Anna1945
    Participant
    Rank: Rank 7

    Endorsement:
    Over a period of several weeks, tried to post requests, never made it.
    On one Sunday alone, spent over ninety minutes just on images.
    Within 15 minutes of using this, posted my first request.
    At that time I thanked the author and also mentioned to
    Warrant 13 that aspects might be worth looking into.
    Once again my thanks to the author.....continued success!!!!

    #2065017
    Lewd18Ratio
    Participant
    Rank: Rank-2

    Thank you very much for taking the time to share your experience.
    I'm deeply grateful for your endorsement! It's wonderful to know that the tool has been helpful and made posting requests easier for you. Your encouragement motivates me to keep enhancing the tool. Thank you for your trust and support!

    #2065024
    Anna1945
    Participant
    Rank: Rank 7

    Well deserved!!!!
    Good luck!!!

    #2066506
    Lewd18Ratio
    Participant
    Rank: Rank-2

    Changelog Announcement: DazArtificer v1.7 - Big Update

    Introducing DIM Creator to DazArtificer!

    I'm excited to announce a significant update to DazArtificer, version 1.7, which marks the inclusion of the DIM Creator, a tool I had initially planned to release separately. However, considering that DazArtificer itself is costing 500 points, I decided it only fair that you all get the full suite of features without extra charges. So, here's what's new:

    • DIM Creator: A new tool for creating custom DIM files, facilitating easy installation and uninstallation of models with the DAZ Install Manager.
    • Launch DIM Creator Button: Seamlessly transition from DazArtificer to DIMCreator, enhancing workflow and integration.
    • DIM Creator Module: DIMCreator is now integrated as a module within DazArtificer, allowing for direct packaging and management of digital content.
    • DazArtificer Folder in Documents: A dedicated folder for all DazArtificer files and configurations, ensuring organized and easy access.
    • Configuration Management: Customizable settings and preferences stored in a new Documents folder for a more personalized experience.

    This update not only enhances DazArtificer's capabilities but also underscores my commitment to providing comprehensive tools that cater to the needs of the 3D modeling and rendering community. Enjoy the new features!

    DazArtificer Page Here

    v1.7 DazArtificer

    #2066646
    WeAllDeserve2Dice
    Participant
    Rank: Rank 4

    Is it every update 500 points? (sorry if this was already asked)

    #2066655
    Grouchy Old Fart
    Participant
    Rank: Rank 3

    ..a DIM builder?

    You had my interest - now you have my attention.

    One of the most frustrating things I've found is to take a perfect scene and scrape out all the files I need to re-create it (that aren't default Genesis mesh stuff) and make an archive of just those files. If this can make a DIM and an archive of files (that is, a bunch of files in a separate directory so I can make a .zip of just those files), I'll be on it in a heartbeat.

    #2066737
    Anna1945
    Participant
    Rank: Rank 7

    Downloaded as soon as appeared.
    Thank you again and congratulations
    Best of luck!!!!

    #2066930
    TheDamnThing
    Participant
    Rank: Rank 3

    This new 1.7 update with DIM creator is simply amazing! It works like a charm. Thank you so much <3

    #2067191
    Lewd18Ratio
    Participant
    Rank: Rank-2

    Hi @wealldeserve2dice thanks for the question, I don't know how the system of the posts works, but my intention is only 1 time 500 Points. The Download link always stays the same and doesn't change. You can select also older versions.

    So keeping it short, 1 time pay, getting also future version

Viewing 15 posts - 16 through 30 (of 49 total)
  • You must be logged in to reply to this topic.

 

Post You Might Like