Automatic sitemap detection (WordPress core, Yoast SEO, Rank Math, All in One SEO, SEOPress) and backup of any existing robots.txt \/ llms.txt before it is replaced in physical mode.<\/li>\n<\/ul>\n\nPrivacy<\/h4>\n\n
The plugin does not send any data to a third-party service. Bot traffic statistics are stored locally only (aggregated by bot and by day, without storing raw IP addresses or user-agent strings) and can be disabled or reset at any time from the settings.<\/p>\n\n
Visits coming from AI assistants are counted the same way: only a counter per day and per assistant, and a counter per landing path, detected from the Referer header or the utm_source<\/code> parameter. No IP address, user-agent or cookie is stored; data older than 30 days is deleted. This tracking can be disabled in Settings > Performance. Pages served by a full-page cache do not run PHP, so part of these visits is not counted.<\/p>\n\n\n\n- Install and activate the plugin from your WordPress admin, or upload the
mephivio-robots-llms<\/code> folder to \/wp-content\/plugins\/<\/code>.<\/li>\n- Go to the Mephivio Robots LLMs<\/strong> menu in your admin.<\/li>\n
- Review the settings (indexed content types, thresholds, write mode) then click Regenerate now<\/strong>.<\/li>\n
- Check the result in the Live preview<\/strong> tab, then visit
\/llms.txt<\/code>, \/llms-full.txt<\/code> and \/robots.txt<\/code> on your site.<\/li>\n<\/ol>\n\n\n\nDoes the plugin write files to my server?<\/h3><\/dt>\nBy default, no: \"Dynamic\" mode serves the files virtually (the same way WordPress natively serves robots.txt<\/code>), without writing anything to disk. \"Physical\" and \"Hybrid\" modes are available if you prefer real files at the root of your site.<\/p><\/dd>\nIs the plugin compatible with Yoast SEO \/ Rank Math \/ All in One SEO?<\/h3><\/dt>\nYes. If the \"Respect SEO noindex\" option is enabled, any content marked noindex<\/code> by one of these plugins is automatically excluded from the generated files.<\/p><\/dd>\nIs the plugin compatible with WooCommerce?<\/h3><\/dt>\nYes, published products are included automatically (description, attributes, categories, SKU), and cart \/ checkout \/ account pages are excluded by default.<\/p><\/dd>\n
How do the bot statistics work?<\/h3><\/dt>\nWhen a known bot requests llms.txt<\/code>, llms-full.txt<\/code> or robots.txt<\/code>, the plugin classifies its user-agent and stores an aggregated counter. To protect performance, increments are rate-limited per bot\/document. No IP address or raw user-agent is kept. History is kept on a rolling 30-day window.<\/p><\/dd>\nWhat happens when I uninstall the plugin?<\/h3><\/dt>\nBy default, nothing is deleted automatically (your settings remain in the database as long as the plugin is only deactivated). A dedicated \"Delete my data on uninstall\" option (unchecked by default) allows you to fully clean up the plugin's settings, cache and metadata when it is permanently removed. Any physical files that may have been written (llms.txt<\/code>, llms-full.txt<\/code>, robots.txt<\/code>) are never deleted automatically.<\/p><\/dd>\nWhy are the global rules repeated for Googlebot, GPTBot\u2026?<\/h3><\/dt>\nUnder RFC 9309, a crawler only follows the most specific group matching its user-agent and ignores User-agent: *<\/code>. Since 1.4.0 the global Disallow rules are therefore repeated in every \"Allow\" group, so they apply to every listed crawler. Custom rules written before any User-agent:<\/code> line are handled the same way.<\/p><\/dd>\nHow do the Markdown pages work?<\/h3><\/dt>\nWhen \"Markdown pages\" is enabled, every page present in llms.txt is also available with .md<\/code> appended to its URL (https:\/\/example.com\/my-page.md<\/code>). Excluded content is never served. The Markdown responses send X-Robots-Tag: noindex<\/code> and a canonical link to the HTML page.<\/p><\/dd>\nWhat happens to my existing robots.txt in physical mode?<\/h3><\/dt>\nBefore replacing a robots.txt or llms.txt file that was not written by the plugin, a copy is saved in wp-content\/uploads\/mephivio-robots-llms\/backups\/<\/code>. The Diagnostics tab lists these backups.<\/p><\/dd>\n\n<\/dl>\n\n\n1.5.0<\/h4>\n\n\n- Large content\/index caches are now segmented into small non-autoloaded options for better scalability on shared hosting.<\/li>\n
- AI referral statistics use per-day storage, a short write lock and true 30-day retention for landing pages.<\/li>\n
- Front-end fallback HTTP extraction is limited to 2 MB before DOM parsing.<\/li>\n
- Uninstall cleanup covers all new segmented and daily options.<\/li>\n<\/ul>\n\n
1.4.0<\/h4>\n\n\n- Fix: the Dashboard crashed on PHP 8 (fatal ValueError) while a background generation was running (broken format string since 1.3.6).<\/li>\n
- Fix (SEO): robots.txt groups now repeat the global rules. Previously Googlebot, Bingbot, GPTBot\u2026 only received \"Allow: \/\" and ignored the Disallow rules (wp-admin, internal search, cart, checkout\u2026), as specified by RFC 9309.<\/li>\n
- Fix: \/llms.txt, \/llms-full.txt and \/llms-schema.json no longer redirect (301) to a trailing-slash URL.<\/li>\n
- Fix: disabled, empty or unknown documents (for example \/llms-zz.txt) now return a real 404 instead of the front page.<\/li>\n
- Fix: in physical\/hybrid mode, files are now rewritten when their content changes (previously only on a settings save), and any existing file not written by the plugin is backed up before being replaced.<\/li>\n
- Fix: sitemaps are detected (WordPress core when enabled, Yoast SEO, Rank Math, All in One SEO, SEOPress) instead of always declaring \/wp-sitemap.xml.<\/li>\n
- Fix: saving the settings no longer runs a full synchronous regeneration; the interface language applies immediately and regeneration runs in the background (WP-Cron) only when a setting affecting the documents changed.<\/li>\n
- Fix: WooCommerce settings are kept when the settings are saved while WooCommerce is inactive.<\/li>\n
- Fix: custom robots rules written before any User-agent line now apply to every allowed crawler instead of being attached to the last group.<\/li>\n
- Fix: the log directory is protected by an index.html file (no PHP file is written anymore).<\/li>\n
- Fix: PHP 8.4 deprecation in the CSV export; \"Tested up to\" removed from the main plugin header (Plugin Check).<\/li>\n
- New: the \"URL exclusion patterns\" can be kept out of robots.txt (option, enabled by default to keep the previous behavior); duplicated Disallow lines removed.<\/li>\n
- New: per-bot AI crawler control grouped by purpose, updated AI crawler list (Claude-User, Meta-ExternalAgent, Meta-ExternalFetcher, MistralAI-User, DuckAssistBot, Google-CloudVertexBot) and
mephroll_ai_bots<\/code> filter.<\/li>\n- New: Markdown version of indexed pages (
\/my-page.md<\/code>, opt-in).<\/li>\n- New: llms.txt \"## Optional\" section and taxonomy archive sections.<\/li>\n
- New: \"LLM visibility\" panel in the block editor with a custom LLM summary (also available in the classic meta box).<\/li>\n
- New: WooCommerce price and availability in llms-full.txt, JSON-LD (
Offer<\/code>) and Markdown pages.<\/li>\n- New: daily charts of bot requests and aggregated count of visits coming from AI assistants.<\/li>\n
- New: configuration check in the Diagnostics tab and warnings when another plugin changes robots.txt.<\/li>\n
- New: exclusion reasons are displayed as readable, translated labels.<\/li>\n
- New filters:
mephroll_ai_bots<\/code>, mephroll_sitemaps<\/code>, mephroll_ai_referral_sources<\/code>, mephroll_taxonomy_terms_limit<\/code>.<\/li>\n<\/ul>\n\n1.3.8<\/h4>\n\n\n- Updated the plugin Author URI to https:\/\/www.mephivio.fr.<\/li>\n
- Updated version metadata and documentation for the 1.3.8 release.<\/li>\n<\/ul>\n\n
1.3.7<\/h4>\n\n\n- WordPress.org review follow-up: removed the remaining global bootstrap function, prefixed template\/uninstall variables, optimized legacy metadata migration to avoid meta-key queries, and documented intentional WPML\/core hook usage.<\/li>\n<\/ul>\n\n
1.3.6<\/h4>\n\n\n- WordPress.org review: moved all admin CSS and JavaScript output to properly enqueued assets; removed inline
<style><\/code> and <script><\/code> output.<\/li>\n- Collision prevention: replaced the former three-letter code, storage, action, script\/style and metadata prefixes with the unique
mephroll<\/code> \/ MEPHROLL<\/code> prefix.<\/li>\n- Upgrade safety: added a one-time migration for settings, caches, cron state and per-content metadata created by versions up to 1.3.5.<\/li>\n
- Directory packaging: renamed the runtime admin icon so it is not confused with a WordPress.org directory asset, and removed unused directory-style assets from the plugin package.<\/li>\n
- Internationalization packaging: removed bundled PO\/MO\/POT files; WordPress.org can provide community translations while the plugin-specific English\/French interface selector keeps its built-in French mapping.<\/li>\n
- Documentation: clarified the plugin\u2019s distinct scope around multilingual generation, WooCommerce, diagnostics, robots controls, content auditing and local bot statistics.<\/li>\n<\/ul>\n\n
1.3.5<\/h4>\n\n\n- Fix: Place the WooCommerce product-count translator comment immediately before the translatable string so WordPress Plugin Check recognizes it correctly.<\/li>\n<\/ul>\n\n
1.3.4<\/h4>\n\n\n- Fixed the Plugin Check internationalization error for the WooCommerce published-product counter.<\/li>\n
- Added the required translators comment for the localized numeric placeholder.<\/li>\n<\/ul>\n\n
1.3.3<\/h4>\n\n\n- Fixed WooCommerce catalog detection for existing stores and installations where WooCommerce was enabled after the plugin.<\/li>\n
- Added a dedicated, default-enabled WooCommerce product inclusion switch with a detected-product count.<\/li>\n
- Improved product-save regeneration and WooCommerce load-order compatibility.<\/li>\n<\/ul>\n\n
1.3.2<\/h4>\n\n\n- WordPress.org compliance: fixed all issues reported by Plugin Check.<\/li>\n
- Security: explicitly sanitize the submitted Settings array before processing it.<\/li>\n
- Compatibility: replaced
strip_tags()<\/code> with wp_strip_all_tags()<\/code> and removed discouraged manual translation loading.<\/li>\n- Uninstall: clean post metadata through the WordPress metadata API instead of direct SQL queries.<\/li>\n
- Admin: read display query parameters without processing
$_GET<\/code> directly in the view.<\/li>\n- CSV: removed the explicit close on the
php:\/\/output<\/code> stream, which is automatically closed at request end.<\/li>\n- Documentation: declared WordPress 7.1 compatibility and shortened upgrade notices.<\/li>\n<\/ul>\n\n
1.3.1<\/h4>\n\n\n- Fixed: the custom Mephivio icon in the WordPress admin menu is now explicitly constrained to the native 20 \u00d7 20 px menu size, preventing oversized icons when another admin theme or plugin overrides image dimensions.<\/li>\n
- Documentation: updated English and French readmes and changelogs for version 1.3.1.<\/li>\n<\/ul>\n\n
1.3.0<\/h4>\n\n\n- UI: redesigned Settings into clear visual sections for General, LLM files, Content, Robots, Multilingual, Performance and Advanced options.<\/li>\n
- Compatibility: existing setting keys and behavior are unchanged, so upgrades preserve the current configuration.<\/li>\n
- Usability: added section descriptions plus Save Settings actions at the top and bottom of the screen.<\/li>\n
- Responsive admin: improved the Settings layout for tablets and mobile devices.<\/li>\n
- Documentation: updated English and French readmes and changelogs for version 1.3.0.<\/li>\n<\/ul>\n\n
1.2.0<\/h4>\n\n\n- New: plugin-specific admin interface language selector in Settings with English and French options.<\/li>\n
- English is now the default Mephivio Robots LLMs interface language, independently of the WordPress user\/site locale.<\/li>\n
- Switching the plugin interface language does not change the WordPress dashboard language or the public site language.<\/li>\n
- Improved: remaining French source labels in Settings were normalized to English source strings and translated cleanly.<\/li>\n
- Documentation: English and French readmes and changelogs updated for version 1.2.0.<\/li>\n<\/ul>\n\n
1.1.1<\/h4>\n\n\n- Security: real-page rendering now validates same-site URLs and uses
wp_safe_remote_get()<\/code> with TLS verification enabled.<\/li>\n- Security\/performance: bot statistics writes are rate-limited to reduce database-write abuse from spoofed public user-agents.<\/li>\n
- Reliability: fixed resumable indexing so a time-budget interruption in the middle of a page resumes at the correct item instead of duplicating earlier items.<\/li>\n
- Reliability: added a short-lived generation lock to avoid concurrent cron\/admin batches corrupting progress.<\/li>\n
- Privacy\/SEO safety: generated
robots.txt<\/code> now respects WordPress' \"Discourage search engines\" setting and never replaces the core private-site policy in dynamic mode.<\/li>\n- Performance: activation and upgrades now schedule indexing in the background instead of running an 8-second batch inside the activation request.<\/li>\n
- Compatibility: physical file writes and log writes now use the WordPress Filesystem API and fail gracefully when direct filesystem access is unavailable.<\/li>\n
- Fixed: WooCommerce products can now actually be unchecked after their default selection.<\/li>\n
- Fixed: removed a broken admin icon reference and use a WordPress Dashicon instead.<\/li>\n
- Validation: numeric settings are now clamped to supported ranges.<\/li>\n<\/ul>\n\n
1.1.0<\/h4>\n\n\n- Fix: activation could hang \/ time out on sites with a large number of posts.<\/strong> Content indexing now runs as a time-budgeted, resumable batch process (about 8 seconds per run by default, filterable via
mephroll_batch_time_budget<\/code>) instead of a single blocking pass over the entire catalog. If your content library is too large to process in one go, generation now continues automatically in the background via WP-Cron every ~10 seconds until finished, while previously published files keep being served in the meantime.<\/li>\n- This fix applies uniformly to plugin activation, plugin updates, the \"Regenerate now\" button, and automatic regeneration on post save\/delete \u2014 all now go through the same safe batched engine.<\/li>\n
- New: a \"Background generation in progress\u2026\" notice with a processed\/estimated-total counter is shown on the Dashboard while a batch is running, with automatic page refresh.<\/li>\n
- New: a \"Reset generation\" button (Diagnostics tab) to clear a stuck batch state (e.g. if WP-Cron is not running on the site) without affecting already-published files.<\/li>\n
- New: a stall warning is shown if a background generation hasn't progressed for more than 10 minutes, usually indicating WP-Cron isn't executing on the site.<\/li>\n<\/ul>\n\n
1.0.0<\/h4>\n\n\n- Full rename of the plugin (formerly \"Flappy \/ Master Robots LLMs\") to Mephivio Robots LLMs<\/strong>, ahead of its free release on WordPress.org.<\/li>\n
- New: JSON-LD feed generation (
llms-schema.json<\/code>, schema.org ItemList<\/code>) alongside llms.txt\/llms-full.txt.<\/li>\n- New: Polylang \/ WPML multilingual compatibility \u2014 generates one document per detected language (
llms-{lang}.txt<\/code>, llms-full-{lang}.txt<\/code>).<\/li>\n- New: aggregated, anonymized statistics of known bot requests on the published files (\"Bot statistics\" tab), can be disabled.<\/li>\n
- New:
uninstall.php<\/code> with optional cleanup (disabled by default) of the plugin's options and metadata.<\/li>\n- Security: fixed CSV Formula Injection in the exclusions export.<\/li>\n
- Security: replaced
maybe_unserialize()<\/code> with hardened deserialization (allowed_classes => false<\/code>) on content metadata coming from arbitrary keys, removing any PHP Object Injection risk.<\/li>\n- Security\/Performance: the real HTML rendering fallback (outbound HTTP request to the page itself, useful for some page builders such as Divi 5) is now disabled by default and becomes an explicit opt-in setting (\"Real HTML rendering (advanced)\").<\/li>\n
- Reliability: dynamic rendering of llms.txt \/ llms-full.txt \/ llms-schema.json now respects their individual enable\/disable settings (previously, only robots.txt did).<\/li>\n
- Full internationalization of the admin interface (translatable strings via the
mephivio-robots-llms<\/code> text domain), with a bundled French translation.<\/li>\n- Removed legacy assets and commercial documentation inherited from another, unrelated product.<\/li>\n<\/ul>\n\n
History (under the name Master\/Flappy Robots LLMs)<\/h4>\n\n\n- 2.6.8 \u2014 Formatted llms.txt entries as valid Markdown links compliant with the llmstxt.org spec, grouped entries by section (Posts, Pages, Products\u2026), guaranteed an H1 title.<\/li>\n
- 2.6.7 \u2014 Removed internal scores from the public llms.txt \/ llms-full.txt exports (kept in the admin only).<\/li>\n
- 2.6.5 \u2014 WooCommerce compatibility (automatic inclusion of the product catalog, exclusion of cart\/checkout\/account pages).<\/li>\n
- 2.6.1 \u2014 Security audit: sanitized the URL tester, moved the log file out of a publicly accessible directory, filtered custom robots.txt rules.<\/li>\n<\/ul>","raw_excerpt":"Generates llms.txt, llms-full.txt, JSON-LD and robots.txt discovery files for AI agents and search crawlers, without promising ranking gains.","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/es-pr.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin\/361872","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/es-pr.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin"}],"about":[{"href":"https:\/\/es-pr.wordpress.org\/plugins\/wp-json\/wp\/v2\/types\/plugin"}],"replies":[{"embeddable":true,"href":"https:\/\/es-pr.wordpress.org\/plugins\/wp-json\/wp\/v2\/comments?post=361872"}],"author":[{"embeddable":true,"href":"https:\/\/es-pr.wordpress.org\/plugins\/wp-json\/wporg\/v1\/users\/mephivio34"}],"wp:attachment":[{"href":"https:\/\/es-pr.wordpress.org\/plugins\/wp-json\/wp\/v2\/media?parent=361872"}],"wp:term":[{"taxonomy":"plugin_section","embeddable":true,"href":"https:\/\/es-pr.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_section?post=361872"},{"taxonomy":"plugin_tags","embeddable":true,"href":"https:\/\/es-pr.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_tags?post=361872"},{"taxonomy":"plugin_category","embeddable":true,"href":"https:\/\/es-pr.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_category?post=361872"},{"taxonomy":"plugin_contributors","embeddable":true,"href":"https:\/\/es-pr.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_contributors?post=361872"},{"taxonomy":"plugin_business_model","embeddable":true,"href":"https:\/\/es-pr.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_business_model?post=361872"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}