watchdog · PyPI

Project description

Python API and shell utilities to monitor file system events.

Works on 3.9+.

Example API Usage

A simple program that uses watchdog to monitor directories specified as command-line arguments and logs events generated:

import timefrom watchdog.events import FileSystemEvent, FileSystemEventHandlerfrom watchdog.observers import Observerclass MyEventHandler(FileSystemEventHandler):    def on_any_event(self, event: FileSystemEvent) -> None:        print(event)event_handler = MyEventHandler()observer = Observer()observer.schedule(event_handler, ".", recursive=True)observer.start()try:    while True:        time.sleep(1)finally:    observer.stop()    observer.join()

Shell Utilities

Watchdog comes with an optional utility script called watchmedo. Please type watchmedo --help at the shell prompt to know more about this tool.

Here is how you can log the current directory recursively for events related only to *.py and *.txt files while ignoring all directory events:

watchmedo log \
    --patterns='*.py;*.txt' \
    --ignore-directories \
    --recursive \
    --verbose \
    .

You can use the shell-command subcommand to execute shell commands in response to events:

watchmedo shell-command \
    --patterns='*.py;*.txt' \
    --recursive \
    --command='echo "${watch_src_path}"' \
    .

Please see the help information for these commands by typing:

watchmedo [command] --help

About watchmedo Tricks

watchmedo can read tricks.yaml files and execute tricks within them in response to file system events. Tricks are actually event handlers that subclass watchdog.tricks.Trick and are written by plugin authors. Trick classes are augmented with a few additional features that regular event handlers don’t need.

An example tricks.yaml file:

tricks:- watchdog.tricks.LoggerTrick:
    patterns: ["*.py", "*.js"]- watchmedo_webtricks.GoogleClosureTrick:
    patterns: ['*.js']
    hash_names: true
    mappings_format: json                  # json|yaml|python
    mappings_module: app/javascript_mappings
    suffix: .min.js
    compilation_level: advanced            # simple|advanced
    source_directory: app/static/js/
    destination_directory: app/public/js/
    files:
      index-page:
      - app/static/js/vendor/jquery*.js
      - app/static/js/base.js
      - app/static/js/index-page.js
      about-page:
      - app/static/js/vendor/jquery*.js
      - app/static/js/base.js
      - app/static/js/about-page/**/*.js

The directory containing the tricks.yaml file will be monitored. Each trick class is initialized with its corresponding keys in the tricks.yaml file as arguments and events are fed to an instance of this class as they arrive.

Installation

Install from PyPI using pip:

$ python -m pip install -U watchdog# or to install the watchmedo utility:
$ python -m pip install -U "watchdog[watchmedo]"

Install from source:

$ python -m pip install -e .# or to install the watchmedo utility:
$ python -m pip install -e '.[watchmedo]'

Documentation

You can browse the latest release documentation online.

Contribute

Fork the repository on GitHub and send a pull request, or file an issue ticket at the issue tracker. For general help and questions use stackoverflow with tag python-watchdog.

Create and activate your virtual environment, then:

python -m pip install tox
python -m tox [-q] [-e ENV]

If you are making a substantial change, add an entry to the “Unreleased” section of the changelog.

Supported Platforms

• Linux 2.6 (inotify)
• macOS (FSEvents, kqueue)
• FreeBSD/BSD (kqueue)
• Windows (ReadDirectoryChangesW with I/O completion ports; ReadDirectoryChangesW worker threads)
• OS-independent (polling the disk for directory snapshots and comparing them periodically; slow and not recommended)

Note that when using watchdog with kqueue, you need the number of file descriptors allowed to be opened by programs running on your system to be increased to more than the number of files that you will be monitoring. The easiest way to do that is to edit your ~/.profile file and add a line similar to:

ulimit -n 1024

This is an inherent problem with kqueue because it uses file descriptors to monitor files. That plus the enormous amount of bookkeeping that watchdog needs to do in order to monitor file descriptors just makes this a painful way to monitor files and directories. In essence, kqueue is not a very scalable way to monitor a deeply nested directory of files and directories with a large number of files.

About using watchdog with editors like Vim

Vim does not modify files unless directed to do so. It creates backup files and then swaps them in to replace the files you are editing on the disk. This means that if you use Vim to edit your files, the on-modified events for those files will not be triggered by watchdog. You may need to configure Vim appropriately to disable this feature.

About using watchdog with CIFS

When you want to watch changes in CIFS, you need to explicitly tell watchdog to use PollingObserver, that is, instead of letting watchdog decide an appropriate observer like in the example above, do:

from watchdog.observers.polling import PollingObserver as Observer

Dependencies

1. Python 3.9 or above.
2. XCode (only on macOS when installing from sources)
3. PyYAML (only for watchmedo)

Licensing

Watchdog is licensed under the terms of the Apache License, version 2.0.

• Copyright 2018-2024 Mickaël Schoentgen & contributors
• Copyright 2014-2018 Thomas Amland & contributors
• Copyright 2012-2014 Google, Inc.
• Copyright 2011-2012 Yesudeep Mangalapilly

Project source code is available at Github. Please report bugs and file enhancement requests at the issue tracker.

Why Watchdog?

Too many people tried to do the same thing and none did what I needed Python to do:

• pnotify
• unison fsmonitor
• fsmonitor
• guard
• pyinotify
• inotify-tools
• jnotify
• treewatcher
• file.monitor
• pyfilesystem

The watchdog Python module is a powerful library designed for monitoring file system events. It provides a robust API and shell utilities to observe changes in directories and files across different platforms. Here's a comprehensive overview of the watchdog module:

Core Functionality

Watchdog's primary purpose is to monitor file system events, such as file creation, modification, deletion, and movement. It works on Python 3.9 and later versions[3].

Key Components

1. Observer: The central component of watchdog is the Observer class. It's responsible for watching directories and dispatching events to event handlers[4].
2. Events: These represent various file system occurrences, including file creation, modification, deletion, and movement[4].
3. Handlers: Event handlers process the events detected by the Observer. Watchdog provides several types of handlers:
• LoggingEventHandler
• FileSystemEventHandler
• PatternMatchingEventHandler[4]

Installation

You can easily install watchdog using pip:

python -m pip install -U watchdog

To include the watchmedo utility, use:

python -m pip install -U "watchdog[watchmedo]"

Basic Usage

Here's a simple example of how to use watchdog:

import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class MyHandler(FileSystemEventHandler):
    def on_any_event(self, event):
        print(f"Event occurred: {event}")

if __name__ == "__main__":
    event_handler = MyHandler()
    observer = Observer()
    observer.schedule(event_handler, path=".", recursive=True)
    observer.start()

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

This script monitors the current directory for any changes and prints out the events as they occur[1][3].

Advanced Features

1. Pattern Matching: Watchdog allows you to monitor specific file types using pattern matching. For example, you can watch only for changes to .csv files[2].
2. Recursive Monitoring: You can monitor subdirectories recursively, allowing for comprehensive file system observation[1].
3. Cross-platform Support: Watchdog works across different operating systems, providing a consistent API for file system monitoring[1].
4. Shell Utility - watchmedo: This optional command-line tool comes with watchdog, allowing you to monitor directories and execute shell commands in response to events[3].
5. Custom Event Handling: You can create custom event handlers to perform specific actions when certain events occur, such as backing up files or logging user activities[4].

Use Cases

1. Automated Backups: Trigger backups when files are modified or created.
2. Development Tools: Auto-reload applications during development when source files change.
3. Security Monitoring: Log file system activities for security purposes.
4. Data Processing: Automatically process new files as they are added to a directory.

Considerations

1. Performance: For high-performance scenarios, especially on network drives or large file systems, you might need to use the PollingObserver instead of the default Observer[4].
2. CIFS Compatibility: When using watchdog with CIFS (Common Internet File System), you need to explicitly use PollingObserver for proper functionality[3].
3. Resource Usage: Be mindful of resource usage, especially when monitoring large directories or using recursive monitoring.

Watchdog provides a flexible and powerful solution for file system monitoring in Python, suitable for a wide range of applications from simple file watching to complex automated workflows triggered by file system events.

Citations: [1] https://python-watchdog.readthedocs.io/en/stable/ [2] https://www.geeksforgeeks.org/create-a-watchdog-in-python-to-look-for-filesystem-changes/ [3] https://pypi.org/project/watchdog/ [4] https://blog.jcharistech.com/2023/01/31/introduction-to-python-watchdog-monitoring-file-systems-and-directories/