Internals
Subject to Change
These are just notes over how the internals of AppDaemon work, but the implementation details are subject to change.
These notes are intended to assist anyone that wants to understand AppDaemon’s internals better.
Structure
The Python project follows the conventional PEP 621, using a pyproject.toml file to define its metadata.
The repository is divided into various folder:
./appdaemonsource code of the Python package
./docssource code from which this documentation is built
./teststests written with
pytest./confconfiguration directory, containing some sample files
AppDaemon is organized into several subsystems managed by the central AppDaemon class. Each subsystem handles a specific aspect of functionality:
Core Subsystems
App Management (
AppManagement) - Provides the mechanics to manage the lifecycle of user apps, which includes tracking the user files for changes and reloading apps as needed.Scheduler (
Scheduler) - Provides time-based scheduling, sunrise/sunset calculations, and time travel functionality for testing.State (
State) - Tracks and manages entity states across different namespaces, provides state persistence between AppDaemon runs, and handles callbacks based on state changes.Events (
Events) - Provides the mechanics to fire events, hand events off to relevant plugins (such as HASS for Home Assistant), and process event callbacksCallbacks (
Callbacks) - Container for storing and managing all registered callbacks from apps.Services (
Services) - Provides the mechanics to register/deregister services and allows apps to call them with a<domain>/<service_name>string.Plugin Management (
PluginManagement) - Manages external plugins that provide connectivity to external systems (Home Assistant, MQTT, etc.).
Threading & Async
Threading (
Threading) - Manages worker threads for executing app callbacks and handles thread pinning.Thread Async (
ThreadAsync) - Bridges between threaded callback execution and the main async event loop using queues.Utility Loop (
Utility) - Runs periodic maintenance tasks like file change detection, thread monitoring, and performance diagnostics.
Additional Subsystems
Sequences (
Sequences) - Manages configurable automation sequences with steps like service calls, waits, and conditionals.Futures (
Futures) - Handles asynchronous operations and provides mechanisms for cancelling long-running tasks.
Reference
- class appdaemon.appdaemon.AppDaemon(logging: Logging, loop: AbstractEventLoop, ad_config_model: AppDaemonConfig, exit_stack: ExitStack | None = None)
Top-level container for the subsystem objects. This gets passed to the subsystem objects and stored in them as the
self.ADattribute.Asyncio:
Subsystems:
Attribute
Object
app_managementcallbackseventsfutureshttppluginsPluginsschedulerservicessequencesSequencesstatethreadingThreadingutility- property apps_enabled
Flag for whether
disable_appswas set in the AppDaemon config
- property config_dir
Path to the AppDaemon configuration files. Defaults to the first folder that has
./apps~/.homeassistant/etc/appdaemon
- executor: ThreadPoolExecutor
Executes functions from a pool of async threads. Configured with the
threadpool_workerskey. Defaults to 10.
- loop: AbstractEventLoop
Main asyncio event loop
- register_http(http: HTTP)
Sets the
self.httpattribute with aHTTPobject and starts the admin loop.
- start() None
Start AppDaemon, which also starts all the component subsystems like the scheduler, etc.
Note: The scheduler is started by the utility loop after plugins are ready.
- async stop() None
Stop AppDaemon by calling the stop method of the subsystems.
This does not stop the event loop, but waits for all the existings tasks to finish before returning, which has a 3s timeout.
PluginsState
- class appdaemon.app_management.AppManagement(ad: AppDaemon)
Subsystem container for managing app lifecycles
- add_plugin_object(name: str, object: PluginBase) None
Add the plugin object to the internal dictionary of
ManagedObjects
- app_cfg_rel_path(app_name: str) Path
Get a Path object to the config file for the app, relative to the apps directory.
- property app_config: AllAppConfig
Keeps track of which module and class each app comes from, along with any associated global modules.
- app_module_rel_path(app_obj: object) Path
Get a Path object to the module file for the app, relative to the apps directory.
This uses the loaded python modules form the
sys.modulesdict.
- app_run_context(app: str, **kwargs)
Context manager for running an app to help during testing.
- Parameters:
app (str) – The name of the app to run. Must have an entry in the app_config root.
**kwargs – Arbitrary keyword arguments representing configuration fields to temporarily update the app with.
- async check_app_config_files(update_actions: UpdateActions)
Updates self.mtimes_config and self.app_config
- async check_app_python_files(update_actions: UpdateActions)
Checks the python files in the app directory. Part of self.check_app_updates sequence
- async check_app_updates(plugin_ns: str | None = None, mode: UpdateMode = UpdateMode.NORMAL, update_actions: UpdateActions | None = None) None
Checks the states of the Python files that define the apps, reloading when necessary.
Called as part of
utility_loop.Utility.loop()- NORMAL
Checks for changes and reloads apps as necessary.
- INIT
Used during startup trigger processing the import paths and initializing the dependency manager.
- TERMINATE
Adds all apps to the set to be terminated.
- RELOAD_APPS
Adds all apps and the modules they depend on to the respective reload sets. Used by the app reload service.
- PLUGIN_FAILED
Stops all the apps of a plugin that failed.
- PLUGIN_RESTART
Restarts all the apps of a plugin that has started again.
- TESTING
Testing mode, used during testing to load apps without starting them.
- Parameters:
plugin_ns (str, optional) – Namespace of a plugin to restart, if necessary. Defaults to None.
mode (UpdateMode, optional) – Defaults to
UpdateMode.NORMAL.update_actions (UpdateActions, optional) – The update actions to perform. Defaults to None.
- property config_filecheck: FileCheck
Property that aliases the
FileCheckinstance for the app config files
- async create_app(app: str, **app_config) None
Create an app
- Parameters:
app (str) – The name of the app to create.
- App Config Kwargs:
class (str): The class name of the app to create. module (str): The module where the app class is located. write_app_file(bool, optional): Whether to write the app config to a file. Defaults to True. app_dir (str, optional): The directory to write the app file to, relative to the appdaemon apps directory. Defaults to “ad_apps”. app_file (str, optional): The name of the app file to write, including extension. Defaults to “{app_name}.yaml”.
- async create_app_object(app_name: str) Any | None
Instantiates an app by name and stores it in
self.objects.This does not work on global module apps.
- Parameters:
app_name (str) – Name of the app, as defined in a config file
- Raises:
PinOutofRange – Caused by passing in an invalid value for pin_thread
MissingAppClass – When there’s a problem getting the class definition from the loaded module
AppInstantiationError – When there’s another, unknown error creating the class from its definition
- async edit_app(app: str, **kwargs)
Used to edit an app, which is already in Yaml. It is expecting the app’s name
- filter_files: dict[str, float]
Dictionary of the modified times of the filter files and their paths.
- get_app_config_files() set[Path]
Get a set of valid app fonfig files in the app directory.
Valid files are ones that are readable, not inside an excluded directory, and not starting with a “.” character.
- async get_app_config_files_async() set[Path]
Get a set of valid app config files in the app directory.
Valid files are ones that are readable, not inside an excluded directory, and not starting with a “.” character.
- get_python_files() set[Path]
Get a set of valid Python files in the app directory.
Valid files are ones that are readable, not inside an excluded directory, and not starting with a “.” character.
- async get_python_files_async() set[Path]
Get a set of valid app config files in the app directory.
Valid files are ones that are readable, not inside an excluded directory, and not starting with a “.” character.
- async import_module(module_name: str)
Reads an app into memory by importing or reloading the module it needs
- async increase_active_apps(name: str)
Marks an app as active and updates the sensors for active/inactive apps.
- async increase_inactive_apps(name: str)
Marks an app as inactive and updates the sensors for active/inactive apps.
- objects: dict[str, ManagedObject]
Dictionary of dictionaries with the instantiated apps, plugins, and sequences along with some metadata. Gets populated by
self.init_object, which instantiates the app classesself.init_plugin_objectself.init_sequence_object
- property python_filecheck: FileCheck
Property that aliases the
FileCheckinstance for the app python files
- async read_config_file(file: Path) AllAppConfig
Reads a single YAML or TOML file into a pydantic model. This also sets the
config_pathattribute of any AppConfigs.This function is primarily used by the create/edit/remove app methods that write yaml files.
- reversed_graph: dict[str, set[str]] = {}
Dictionary that maps full module names to sets of those that depend on them
- async start() None
Start the app management subsystem.
This method: * Initializes admin entities * Initializes the dependency manager (INIT mode) * Loads all apps (normal mode)
- async start_app(app_name: str)
Initializes a new object and runs the initialize function of the app.
This does not work on global module apps because they only exist as imported modules.
- Parameters:
app_name (str) – Name of the app to start
- async stop() None
Stop the app management subsystem and all the running apps.
Calls
check_app_updates()withUpdateMode.TERMINATE
- async stop_app(app_name: str, *, delete: bool = False) bool
Stops the app
- Returns:
Whether stopping was successful or not
- Return type:
- async terminate_sequence(name: str) bool
Terminate the sequence.
- Returns:
Whether the sequence was found and terminated
- Return type:
- update_app(app: str, **kwargs)
Update the configuration of a specified app with new keyword arguments.
- Parameters:
app (str) – The name of the app to update.
**kwargs – Arbitrary keyword arguments representing configuration fields to update.
Notes
Dumps the app’s configuration to a dict, merges the kwargs into it, and validates the result.
Unlike edit_app, this method does not write to a file but updates the in-memory configuration.
Logs warnings if the app is not found or if validation fails.
- class appdaemon.callbacks.Callbacks(ad: AppDaemon)
Container for storing callbacks. Modified by
EventsandState- async dump_callbacks()
Dumps info about the callbacks to the
Diaglog
- class appdaemon.dependency_manager.DependencyManager(python_files: dataclasses.InitVar[Iterable[pathlib._local.Path]], config_files: dataclasses.InitVar[Iterable[pathlib._local.Path]])
Keeps track of all the python files and the app config files (either yaml or toml)
Instantiating this class will walk the app_directory with
pathlib.Path.rglobto find all the files. This happens both for app config files and app python files.The main purpose of breaking this out from
AppManagementis to make it independently testable.- dependent_apps(modules: str | Iterable[str], transitive: bool = True) set[str]
Find the apps that are dependent on any of the modules given. This includes the transitive closure of both the module and app dependencies.
- dependent_modules(modules: str | Iterable[str])
Uses
find_all_dependentswith the reversed dependency graph to find the transitive closure of the python module dependencies.
- classmethod from_app_directory(app_dir: Path, exclude: str | Iterable[str] | None = None) DependencyManager
Creates a new instance of the dependency manager from the given app directory
- get_dependent_apps(modules: Iterable[str]) set[str]
Finds all of the apps that depend on the given modules, even indirectly
- class appdaemon.events.EventCallback(*args, **kwargs)
- class appdaemon.events.Events(ad: AppDaemon)
Subsystem container for handling all events
- async add_event_callback(name: str, namespace: str, cb: Callable, event: str | Iterable[str] | None = None, timeout: str | int | float | timedelta | None = None, oneshot: bool = False, pin: bool | None = None, pin_thread: int | None = None, kwargs: dict[str, Any] | None = None) str | list[str] | None
Add an event callback to AppDaemon’s internal dicts.
Uses the internal callback lock to ensure that the callback is added in a thread-safe manner, and adds an entity in the admin namespace to track the callback.
Includes a feature to automatically cancel the callback after a timeout, if specified.
- Parameters:
name (str) – Name of the app registering the callback. This is important because all callbacks have to be associated with an app.
namespace (str) – Namespace to listen for the event in. All events are fired in a namespace, and this will only listen for events in that namespace.
cb (Callable) – Callback function.
timeout (int, optional)
oneshot (bool, optional) – If
True, the callback will be removed after it is executed once. Defaults toFalse.kwargs – List of values to filter on, and additional arguments to pass to the callback.
- Returns:
Noneor the reference to the callback handle.
- async cancel_event_callback(name: str, handle: str, *, silent: bool = False)
Cancels an event callback.
- async fire_event(namespace: str, event: str, **kwargs: Any) dict[str, Any] | None
Fires an event.
If the namespace does not have a plugin associated with it, the event will be fired locally. If a plugin is associated, the firing of the event will be delegated to the plugin, under the understanding that when the event is fired, the plugin will notify appdaemon that it occurred, usually via the system the plugin is communicating with.
- async has_log_callback(name: str)
Returns
Trueif the app has a log callback,Falseotherwise.Used to prevent callback loops. In the calling logic, if this function returns
Truethe resulting logging event will be suppressed.- Parameters:
name (str) – Name of the app.
- async info_event_callback(name: str, handle: str)
Gets the information of an event callback.
- Parameters:
- Raises:
ValueError – an invalid name or handle was provided
- Returns:
A dictionary of callback entries or rise a
ValueErrorif an invalid handle is provided.
- async process_event(namespace: str, data: dict[str, Any])
Processes an event that has been received either locally or from a plugin.
- Parameters:
namespace (str) – Namespace the event was fired in.
data – Data associated with the event.
- Returns:
None.
- async process_event_callbacks(namespace: str, data: dict[str, Any]) None
Processes a pure event callback.
Locate any callbacks that may be registered for this event, check for filters and if appropriate, dispatch the event for further checking and eventual action.
- Parameters:
namespace (str) – Namespace of the event.
data – Data associated with the event.
- Returns:
None.
Custom exceptions used by AppDaemon and helper functions to format them in the logs.
- appdaemon.exceptions.exception_context(logger: Logger, app_dir: Path, header: str | None = None)
Context manager to handle exceptions in a block of code
- class appdaemon.futures.Futures(ad: AppDaemon)
Subsystem container for managing
Futureobjects- add_future(app_name: str, future: asyncio.Future | Future)
Add a future to the registry and a callback that removes itself from the registry after it finishes.
- futures: dict[str, list[asyncio.Future | Future]]
Dictionary of futures registered by app name
- class appdaemon.logging.AppNameFormatter(fmt=None, datefmt=None, style=None)
Logger formatter to add ‘appname’ as an interpolatable field.
- format(record)
Format the specified record as text.
The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.
- class appdaemon.logging.DuplicateFilter(logger: Logger, threshold: float, delay: float, timeout: float)
logging.Filterthat filters duplicate messages
- class appdaemon.logging.LogSubscriptionHandler(ad: AppDaemon, type)
Handle apps that subscribe to logs.
This Handler requires that it’s formatter is an instance of AppNameFormatter.
- emit(record)
Emit a record.
If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.
- class appdaemon.logging.Logging(*args, **kwargs)
Creates and configures the Python logging. The top-level logger is called
AppDaemon. Child loggers are created withget_child().- async add_log_callback(namespace: str, name: str, callback: Callable, level: str | int, pin: bool | None = None, pin_thread: int | None = None, **kwargs) list[str] | None
Adds a callback for log which is called internally by apps.
- Parameters:
- Returns:
Noneor a list of the callback handles, 1 for each logging level above the one given
- get_child(name: str) Logger
Creates a logger with the name
AppDaemon.<name>. Automatically adds aDuplicateFilterwith the config options frommain_log:filter_threshold
filter_repeat_delay
filter_timeout
- Parameters:
name (str) – Child name for the logger.
- Returns:
Child logger
- Return type:
Logger
- get_error() Logger
Gets the top-level error log
- Returns:
Python logger named
Error- Return type:
Logger
- get_logger() Logger
Gets the top-level log
- Returns:
Python logger named
AppDaemon- Return type:
Logger
- async process_log_callbacks(namespace, log_data)
Process Log callbacks
- class appdaemon.plugin_management.PluginBase(ad: AppDaemon, name: str, config: PluginConfig)
Base class for plugins to set up _logging
- property all_namespaces: list[str]
A list of namespaces that includes the main namespace as well as any extra ones.
- class appdaemon.plugin_management.PluginManagement(ad: AppDaemon, config: dict[str, PluginConfig])
Subsystem container for managing plugins
- config: dict[str, PluginConfig]
Config as defined in the appdaemon.plugins section of appdaemon.yaml
- get_plugin_from_namespace(namespace: str) str
Gets the name of the plugin that’s associated with the given namespace.
This function is needed because plugins can have multiple namespaces associated with them.
- plugin_meta: dict[str, dict[str, Any]]
Dictionary storing the metadata for the loaded plugins. {<namespace>: <metadata dict>}
- plugin_objs: dict[str, dict[str, PluginBase | bool | str]]
Dictionary storing the instantiated plugin objects.
{<namespace>: { "object": <PluginBase>, "active": <bool>, "name": <str> }}
- process_meta(meta: dict, name: str)
Looks for certain keys in the metadata dict to override ones in the original AD config. For example, latitude and longitude from a Hass plugin
- async refresh_update_time(plugin_name: str)
Updates the internal time for when the plugin’s state was last updated
- async stop()
Stops all the plugins and clears the callbacks for them.
This needs to be async in order to wait for the apps to all be cleared before stopping the plugins.
- class appdaemon.services.ServiceCallback(*args, **kwargs)
Simple
Protocolfor callbacks for service results.
- class appdaemon.services.Services(ad: AppDaemon)
Subsystem container for handling services
- services
AppDaemon’s internal service registry, which is a set of nested dicts, organized like this:
Namespace
Domain
Service name
Service Info
- services_lock
Re-entrant lock for preventing the service dict from being read and modified at the same time.
- Type:
- class appdaemon.state.State(ad: AppDaemon)
Subsystem container for tracking states
- async add_entity(namespace: str, entity: str, state: Any, attributes: dict | None = None) None
Adds an entity to the internal state registry and fires the
__AD_ENTITY_ADDEDevent
- async add_namespace(namespace: str, writeback: ADWritebackType, persist: bool, name: str | None = None) Path | Literal[False] | None
Add a state namespace.
Fires a
__AD_NAMESPACE_ADDEDevent in theadminnamespace if it’s actually added.- Parameters:
- Returns:
- The path to the namespace database file if added successfully, False if it
already exists, or None if the namespace isn’t persistent.
- Return type:
Path | Literal[False] | None
- async add_persistent_namespace(namespace: str, writeback: ADWritebackType = ADWritebackType.safe) Path | None
Add a namespace that’s stored in a persistent file.
This needs to be an async method to make sure it gets run from the event loop in the main thread. Otherwise, the
DbfilenameShelfcan get messed up because it’s not thread-safe. In some systems, it’ll complain about being accessed from multiple threads, depending on what database driver is used in the background.
- async add_state_callback(name: str, namespace: str, entity: str | None, cb: StateCallback | AsyncStateCallback, timeout: str | int | float | timedelta | None = None, oneshot: bool = False, immediate: bool = False, pin: bool | None = None, pin_thread: int | None = None, kwargs: dict[str, Any] | None = None)
Add a state callback to AppDaemon’s internal dicts.
Uses the internal callback lock to ensure that the callback is added in a thread-safe manner.
- Parameters:
name – Name of the app registering the callback. This is important because all callbacks have to be associated with an app.
namespace – Namespace of the entity to listen to.
entity (str, optional) – Entity ID for listening to state changes. If
None, the callback will be invoked for all state changes in the namespace.cb (StateCallbackType) – Callback function to be invoked when the state changes. Can be sync or async.
oneshot (bool, optional) – If
True, the callback will be removed after it is executed once. Defaults toFalse.immediate (bool, optional) – If
True, the callback will be executed immediately if the entity is already in the new state. Defaults toFalse.kwargs (dict, optional) – Additional parameters arguments to be passed to the callback function.
- Returns:
A string made from
uuid4().hexthat is used to identify the callback. This can be used to cancel the callback later.
- async info_state_callback(handle: str, name: str) tuple[str, str, Any, dict[str, Any]]
Get information about a state callback
Needs to be async to use the callback lock.
- Parameters:
- Returns:
A tuple with the namespace, entity, attribute, and kwargs of the callback
- async periodic_save(interval: str | int | float | timedelta) None
Periodically save all namespaces that are persistent with writeback_type ‘hybrid’
- register_state_services(namespace: str) None
Register the set of state services for the given namespace.
- async remove_entity(namespace: str, entity: str) None
Removes an entity.
If the namespace does not have a plugin associated with it, the entity will be removed locally only. If a plugin is associated, the entity will be removed via the plugin and locally.
- async remove_entity_simple(namespace: str, entity_id: str) None
Used to remove an internal AD entity
Fires the
__AD_ENTITY_REMOVEDevent in a new task
- async remove_namespace(namespace: str) PersistentDict | dict[str, Any] | None
Remove a state namespace. Must not be configured by the appdaemon.yaml file, and must have been added by an app.
Fires an
__AD_NAMESPACE_REMOVEDevent in theadminnamespace if it’s actually removed.
- async remove_persistent_namespace(namespace: str, state: PersistentDict) Path | None
Used to remove the file for a created namespace
- async set_state(name: str, namespace: str, entity: str, _silent: bool = False, *, state: Any | None = None, attributes: dict | None = None, replace: bool = False, **kwargs: Any) dict[str, Any]
- async set_state(name: str, namespace: str, entity: str, _silent: bool = False, **kwargs: Any) dict[str, Any]
Sets the internal state of an entity.
Fires the
state_changedevent under the namespace, and uses relevant plugin objects based on namespace.- Parameters:
name – Only used for a log message
namespace
entity
__silent
state
attributes
replace
- class appdaemon.thread_async.ThreadAsync(ad: AppDaemon)
Module to translate from the thread world to the async world via queues
- stop()
Stops the thread/async loop by putting a sentinel value in the queue.
- class appdaemon.utility_loop.Utility(ad: AppDaemon)
Subsystem container for managing the utility loop
Checks for file changes, overdue threads, thread starvation, and schedules regular state refreshes.
- async _init_loop()
Initialize the utility loop components.
Sets up stats
Starts the web server if configured
Waits for all plugins to initialize
Registers services
Starts the scheduler
Initializes apps if apps are enabled
- _loop_iteration_context() AsyncGenerator[LoopTiming]
Async context manager for running the utility
loop().Contains logic for warnings
Exceptions are logged with tracebacks, but not raised
Warnings are logged if the utility loop takes too long
Handles the timing of the utility loop
- Yields:
LoopTiming – Timing object for recording operation timestamps.
- app_update_event: Event
Event that gets set and cleared
_loop_iteration_context()method, which wraps each iteration of the while loop inloop()
- async loop()
Run the utility loop, which handles the following:
Checking for file changes to update/reload apps if necessary with
check_app_updates()Checking for thread starvation
Checking for overdue threads
Save hybrid namespaces
Gives the plugins a chance to run their own utility functions
Updates performance entities
- async sleep(delay: float, *, timeout_ok: bool)
Sleep for a specified number of seconds.
The purpose of this method is to make sleeping easily and quickly interruptible. This is done by using
asyncio.wait_for()to wait for the stop event to be set, and (usually) ignoring the timeout.