# Apps # Apps management Nextcloud apps can enhance, customize, or restrict the features and experience available to you and your users on the Nextcloud server. In addition to built-in apps such as Files, Activity, and Photos, other apps such as Calendar, Contacts, and Talk can further expand its functionality. After installing the Nextcloud server, you might want to consider enabling, disabling, or restricting some apps for specific groups, depending on your needs and those of your users. ## Apps ![Apps page for enabling and disabling apps.](https://docs.cocobook.com/uploads/images/gallery/2026-10/apps-overview.png) During the Nextcloud server installation, some apps are enabled by default. To see which apps are enabled go to your Apps page. Those apps are supported and developed by Nextcloud GmbH directly and have a **Featured** tag. > **提示** > > Your Nextcloud server needs to be able to communicate with `https://apps.nextcloud.com`, `https://ltd[1-3].nextcloud.com`, `https://garm[1-5].nextcloud.com` to list and download apps. Please make sure to allow these hosts in your firewall or proxy if necessary. > **提示** > > To get access to work-arounds, long-term-support, priority bug fixing and custom consulting for supported apps, contact [Nextcloud GmbH](https://nextcloud.com/enterprise/). > **提示** > > If you would like to develop your own Nextcloud app, you can find out more information in our [developer manual](https://docs.nextcloud.com/server/latest/go.php?to=developer-manual). All apps must be licensed under AGPLv3+ or any compatible license. ## Managing apps ![App store page for installing, enabling and disabling apps.](https://docs.cocobook.com/uploads/images/gallery/2026-10/apps-store.png) You will see which apps are enabled, disabled and available. You'll also see additional app bundles and filters, such as Customization, Security and Monitoring for finding more apps quickly. On the Apps page, you can enable or disable applications. Some apps have configurable options on the Apps page, such as **Enable only for specific groups**, but mainly they are enabled or disabled here, and are configured in your Nextcloud settings (admin and/or user-settings) or in the `config.php`. Select an app to view its description and available configuration options. Clicking the **Enable** button will enable the app. If the app is not part of the Nextcloud installation, it will be downloaded from the app store, installed and enabled. App updates will also be offered to you on this page. Simply click on the **Update** button to update a specific app or use the **Update all** button on top of the page to update all apps. > **提示** > > **Beta releases**: You can also install beta releases of apps directly from here by switching your Nextcloud to the beta channel in the admin overview. ### Update notifications The default `updatenotification` app sends administrators notifications about available app and Nextcloud updates. Moreover, since Nextcloud 29, this app also allows to notify users about updated apps and the changes that are included in the update. This notification is enabled by default if the app provides a changelog. To disable user notifications use: ```console occ config:app:set --type boolean --value="false" updatenotification app_updated.enabled ``` By default. when using the `guests` app, guest users are not notified. To enable notifications for them use: ```console occ config:app:set --type boolean --value="true" updatenotification app_updated.notify_guests ``` ### Managing apps with `occ` In addition to managing apps via the web interface, administrators can enable or disable apps with the `occ` command. To enable an app, use the following command: ```console occ app:enable ``` For example, to enable the "files" app, run: ```console occ app:enable files ``` To enable the app for specific groups, use the `--groups` option: ```console occ app:enable files --groups=admin ``` This command enables the "files" app only for the "admin" group. To disable an app, use: ```console occ app:disable ``` ### Using private API If a third-party app uses private APIs instead of public APIs, installation fails when `'appcodechecker' => true,` is set in `config.php`. ### Using custom app directories Use the **apps_paths** array in `config.php` to set any custom apps directory locations. The key **path** defines the absolute file system path to the app folder. The key **url** defines the HTTP web path to that folder, starting at the Nextcloud web root. The key **writable** indicates if a user can install apps in that folder. Example: To ensure that the default `/apps/` folder only contains apps shipped with Nextcloud, follow this example to set up an `/extra-apps/` folder which will be used to store any additional apps you install: ``` php "apps_paths" => [ [ "path" => OC::$SERVERROOT . "/apps", "url" => "/apps", "writable" => false, ], [ "path" => OC::$SERVERROOT . "/extra-apps", "url" => "/extra-apps", "writable" => true, ], ], ``` > **警告** > > Make sure that the values you choose for `path` and `url` for any custom apps directories do not conflict with directories which already exist in your Nextcloud Server root (installation directory). > **提示** > > Apps paths can be located outside the server root. However, for any **path** outside the server root, you need to create a symbolic link in the server root that points **url** to **path**. For instance, if **path** is `/var/local/lib/nextcloud/extra-apps`, and **url** is `/extra-apps`, then you would use the command `ln` to create the symbolic link like this: `ln -sf /var/local/lib/nextcloud/extra-apps ./extra-apps` ### Configuring the app store timeout The timeout for requests to fetch app store metadata can be configured with the `appstore-timeout` app setting. The value is specified in seconds. This setting is configured with `occ` and is not a `config.php` parameter. For example, to set the timeout to 180 seconds: ```console occ config:app:set settings appstore-timeout --value=180 ``` The default timeout is 120 seconds. To restore the default, remove the custom setting: ```console occ config:app:delete settings appstore-timeout ``` > *Changed in version 33.0.0:* The default app store request timeout was increased from 60 to 120 seconds and can be configured with `occ`. ### Using a self-hosted app store This section explains how to enable installation of apps from a self-hosted app store. At least one of the configured apps directories must be writable. To enable a self-hosted app store: 1. Set the `appstoreenabled` parameter to `true`. This parameter is used to enable the app store in Nextcloud. 2. Set the `appstoreurl` to the URL of your Nextcloud app store. This parameter is used to set the HTTP path to your self-hosted Nextcloud app store. ``` php "appstoreenabled" => true, "appstoreurl" => "https://my.appstore.instance/v1", ``` By default the app store is enabled and configured to use `https://apps.nextcloud.com/api/v1` as the app store URL. Nextcloud will fetch `apps.json` and `categories.json` from there. To use the defaults again, remove the `appstoreenabled` and `appstoreurl` parameters from the configuration. Example: If `categories.json` is available at `https://apps.nextcloud.com/api/v1/categories.json` the app store URL is `https://apps.nextcloud.com/api/v1`. # Apps management API ## Get list of apps Returns a list of apps installed on the Nextcloud server. Authentication is done by sending a Basic HTTP Authorization header. **Syntax: ocs/v1.php/cloud/apps/** - HTTP method: GET - url argument: filter, string - optional (`enabled` or `disabled`) Status codes: - 100 - successful - 101 - invalid input data ### Example $ curl -X GET http://admin:secret@example.com/ocs/v1.php/cloud/apps?filter=enabled -H "OCS-APIRequest: true" - Gets enabled apps ### XML output ``` xml 100 ok files provisioning_api ``` ## Get app info Provides information on a specific application. Authentication is done by sending a Basic HTTP Authorization header. **Syntax: ocs/v1.php/cloud/apps/{appid}** - HTTP method: GET Status codes: - 100 - successful ### Example $ curl -X GET http://admin:secret@example.com/ocs/v1.php/cloud/apps/files -H "OCS-APIRequest: true" - Get app info for the `files` app ### XML output ``` xml 100 ok appinfo/remote.php appinfo/remote.php appinfo/filesync.php files Files File Management AGPL-3.0-or-later Robin Appelman 4.9 true true filesystem ``` ## Enable an app Enable an app. Authentication is done by sending a Basic HTTP Authorization header. **Syntax: ocs/v1.php/cloud/apps/{appid}** - HTTP method: POST Status codes: - 100 - successful ### Example $ curl -X POST http://admin:secret@example.com/ocs/v1.php/cloud/apps/files_texteditor -H "OCS-APIRequest: true" - Enable the `files_texteditor` app ### XML output ``` xml 100 ok ``` ## Disable an app Disables the specified app. Authentication is done by sending a Basic HTTP Authorization header. **Syntax: ocs/v1.php/cloud/apps/{appid}** - HTTP method: DELETE Status codes: - 100 - successful ### Example $ curl -X DELETE http://admin:secret@example.com/ocs/v1.php/cloud/apps/files_texteditor -H "OCS-APIRequest: true" - Disable the `files_texteditor` app ### XML output ``` xml 100 ok ``` # ExApps management - [AppAPI and External Apps](/link/985) - [Deployment configurations](/link/986) - [Managing Deploy Daemons](/link/987) - [Test Deploy Daemon](/link/988) - [Managing ExApps](/link/989) - [Advanced Deploy Options](/link/990) # AppAPI and External Apps Previously, Nextcloud only supported applications written in the PHP programming language. In order to support a wider range of use cases, an ecosystem for **ExApps** (short for "External Apps") was introduced, allowing for the installation of apps as Docker containers. Most of our [Artificial Intelligence](/link/991) (AI) apps are developed as ExApps and thus may require some preparation of your Nextcloud instance before you can install them. > **提示** > > AppAPI and ExApps are optional functionality. If you do not use any non-PHP applications then you may safely disable AppAPI by navigating to the Apps management page in your Nextcloud instance, going to "Your Apps" and clicking the "Disable" button next to AppAPI. ## Installing AppAPI All ExApps require the [AppAPI](https://apps.nextcloud.com/apps/app_api) Nextcloud app as a dependency. As of Nextcloud version 30.0.1, AppAPI is automatically installed by default. If AppAPI is not installed, you can still install it by simply navigating to the Apps management page in your Nextcloud instance and search for AppAPI from the Tools category. ## Setup deploy daemon A Deploy Daemon is the way for Nextcloud to install, communicate with, and control ExApps. > **提示** > > If you are using Nextcloud AIO with the "HaRP" or "Docker Socket Proxy" container enabled, a Deploy Daemon will be automatically created and configured to work out-of-the-box. Otherwise, follow the steps below to set up a Deploy Daemon from the AppAPI admin settings. > **提示** > > After registering a Deploy Daemon, use the **Test Deploy** action to verify it is reachable and working. In the list of Deploy Daemons, click the ... (three‑dots) menu beside the daemon you want to verify and choose Test Deploy. For details on what this check does and how to interpret the results, see [Test Deploy](/link/988). ### HaRP This is the newer and the **recommended** way to install ExApps.\ It requires changes in the proxy of your Nextcloud instance. If you don't have access to the proxy, you can use the usual method [described below](/link/985). 1. Setup a Docker container called [HaRP](https://github.com/nextcloud/HaRP?tab=readme-ov-file#how-to-install-it) that proxies access to Docker and to the ExApps for your Nextcloud instance. Be mindful of changing the values of `HP_SHARED_KEY` and `NC_INSTANCE_URL`, and [generate your own shared key](/link/986) instead of using the example value. 2. Go to AppAPI admin settings. 3. Click on the "Register Daemon" button. 4. A filled form should appear. This default configuration `HaRP Proxy (Host)` should work for most setups. For Nextcloud AIO, use `HaRP All-in-One`.\ If you are using Nextcloud in a custom docker network and would want the HaRP container to be limited to it, use the `HaRP Proxy (Docker)` option to have the fields pre-filled with the common options or change them manually.\ Here, you should ensure the HaRP container itself is launched with the same network as your Nextcloud instance optionally with no ports exposed to the host in step 1, and the same docker network is mentioned in the `Network` field in the deploy config. 5. Ensure the same shared key is used in the HaRP container and in the AppAPI settings. 6. Click "Check connection" to verify that the configuration is correct. 7. Click "Register" to save the Deploy Daemon configuration. 8. Set up a location redirect in your Nextcloud's main proxy configuration to redirect requests to the HaRP container. Some examples for popular reverse proxies can be found in [Configuring Your Reverse Proxy](https://github.com/nextcloud/harp?tab=readme-ov-file#configuring-your-reverse-proxy) in the HaRP readme. 9. Test the whole setup with "Test deploy" in the 3-dots menu of the Deploy Daemon. This is suitable for local setups where the Nextcloud server and the ExApps are on the same machine or in the same docker network. The ExApps in this configuration or the ExApp server need not expose any ExApp related port (23000-23999) necessarily to the host, nor do they need to be reachable from the host. They should be able to reach the HaRP container at the FRP port and the Nextcloud instance. For different/remote setups, see deployment configuration examples [here](/link/986). > **提示** > > The existing ExApps can be migrated to use the new HaRP proxy following [this guide](https://github.com/nextcloud/harp?tab=readme-ov-file#nextcloud-32-migrating-existing-exapps-from-dsp-to-harp). ### Docker Socket Proxy 1. Setup a Docker container called [docker-socket-proxy](https://github.com/nextcloud/docker-socket-proxy#readme) that proxies access to Docker for your Nextcloud instance. 2. Go to the AppAPI admin settings. 3. Click on the "Register Daemon" button. 4. Fill in the required fields: - `Name`: unique name of the Deploy daemon - `Display name`: the name that will be displayed in the UI - `Deployment method`: by default, you will need to choose `docker_install` (`manual_install` is for development or custom use case of manual ExApp installation) - `Daemon Host`: hostname/IP address + port of the Deploy daemon - `Nextcloud URL`: autofilled with current domain, you might need to change the protocol to http/https depending on your setup - `Set as default daemon`: check if you want set new Deploy daemon as default - `Enable https`: check if your Deploy daemon (Docker Socket Proxy) is configured with TLS - Deploy Config: - `Network`: Docker network name, depends on your networking setup, enforces to "host" if "Enable https" is checked - `HaProxy password`: password for Docker Socket Proxy, if it is configured with TLS - `Compute Device`: CPU, CUDA or ROCm, depending on your hardware config on Deploy daemon host machine - `Add additional option` (see [Additional options](/link/987)): setup additional KEY + VALUE deploy config options 5. Click "Check connection" to verify that the configuration is correct. 6. Click "Register" to save the Deploy Daemon configuration. > **提示** > > For remote DSP setup, it should expose the ports on the host. ![AppAPI admin settings showing Deploy Daemon configuration form with Check connection button](https://docs.cocobook.com/uploads/images/gallery/2026-10/app-api-3.png) Deployment configuration examples can be found [here](/link/986). ## Installing ExApps You can now install ExApps from the Nextcloud App Store by clicking "Install" on the respective app in the Apps page. If successful, the ExApp will be displayed under the "Your apps" list. ![Nextcloud Apps page showing installed ExApps in the Your apps list](https://docs.cocobook.com/uploads/images/gallery/2026-10/exapp-list-example.png) ## FAQ - I have two graphics cards (e.g. NVIDIA RTX 3060) with 8 GB of VRAM each. How can I run something which does not fit into one graphics card? - Distributing models across multiple GPUs is currently not supported. You will need a GPU that fits all of the model you are trying to use. - I have a graphics card that does not support CUDA - can I use it and how? - No, our AI apps require GPUs with CUDA support to function at this time. - What is the minimum VRAM size requirement for the GPU if I want to install multiple apps? - When running multiple ExApps on the same GPU, the GPU must hold the largest model amongst the apps you install. - Is it possible to add more graphics cards for my instance to enable parallel requests or to speed up one request? - Parallel processing of AI workloads for the same app with multiple GPUs is currently not supported. - Can I use the CPU and GPU in parallel for AI processing? - No, you can only process AI workloads on either the CPU or GPU for one app. For different apps, you can decide whether to run them on CPU or GPU. - Can I run the same ex app container once for two different nextcloud instances at the same time? - No, each ExApp container can only be used for one Nextcloud instance at a time. You would need to run separate containers and even separate docker hosts for each Nextcloud instance. ## Docker Socket Proxy vs HaRP HaRP can be seen as Docker Socket Proxy version 2.0. It does all what Docker Socket Proxy does, but also addresses the main pain point of ExApps not being reachable by the Nextcloud server (or AppAPI).\ [FRP](https://github.com/fatedier/frp) is used to create a tunnel between the ExApp and the HaRP container so there is no need for the ExApp containers to expose any ports to the host or to be reachable from the Nextcloud server.\ The Nextcloud server can reach the ExApp containers through the HaRP container. HaRP has an additional benefit of being able to proxy requests coming from the Web interface or an API to the ExApp container without being proxied through the Nextcloud server, saving resources, improving performance and supporting additional protocols like WebSockets. HaRP is the recommended way to run ExApps, but if you are not able to use it, Docker Socket Proxy is still supported. Frontend requests in case of Docker Socket Proxy: ``` mermaid graph LR; subgraph Browser A[Frontend] end B[Proxy] subgraph Services behind the proxy C[Docker Socket Proxy] D[ExApp] E[Nextcloud Server / AppAPI] end A --> B B -->|Request to an ExApp| E --Converted to ExApp auth--> D B -->|All other usual requests| E ``` Frontend requests in case of HaRP: ``` mermaid graph LR; subgraph Browser A[Frontend] end B[Proxy] subgraph Services behind the proxy C[HaRP] D[ExApp] E[Nextcloud Server / AppAPI] end B --All other usual requests--> E A --> B B --Direct request to an ExApp--> C --Converted to ExApp auth--> D C --User auth validation--> E ``` # Deployment configurations Currently, two kinds of application deployments are supported: - [Docker Deploy Daemon (Docker Socket Proxy)](/link/986) - [Docker Deploy Daemon (HaRP)](/link/986) ## Docker Deploy Daemon Orchestrates the deployment of applications as Docker containers. > **注意** > > The administrator is responsible for the security actions taken to configure the Docker daemon connected to the Nextcloud instance.\ > These schemes are only examples of possible configurations. > > For Docker Deploy Daemon (HaRP), [AppAPI HaRP](https://github.com/nextcloud/harp) is required.\ > For Docker Deploy Daemon (Docker Socket Proxy), we recommend that you use the [AppAPI Docker Socket Proxy](https://github.com/nextcloud/docker-socket-proxy) or [AIO Docker Socket Proxy](#bkmrk-nextcloud-in-docker-) container for Nextcloud AIO. There are several Docker Daemon Deploy configurations (example schemes): > - Nextcloud and Docker on the **same host** (via socket, DockerSocketProxy, or HaRP) > - Nextcloud on the host and Docker on a **remote** host (via DockerSocketProxy with HTTPS, or HaRP) > - Nextcloud and **ExApps** in the **same Docker network** (via DockerSocketProxy, or HaRP) > - Nextcloud in AIO Docker and **ExApps** in the **same Docker network** (via AIO DockerSocketProxy) ## Docker Deploy Daemon (HaRP) With HaRP, the ExApps initiate the connection for tunneling to the Nextcloud instance and the HaRP container so there is no need to expose any ports or open any firewall rules.\ See the diagrams of the respective configurations in the [Docker Deploy Daemon (Docker Socket Proxy)](/link/986) section below. A little introduction to the default ports of the HaRP container is given below. More about it can be found in the [HaRP's readme](https://github.com/nextcloud/harp?tab=readme-ov-file#environment-variables). - Port `8780` is the HTTP communication port used where Nextcloud connects to the HaRP container. - Port `8781` is the HTTPS communication port when setup. - Port `8782` is the FRP tunnel port used by ExApps to connect to the HaRP container. In any of the cases, the following connections should succeed: - Nextcloud -\> HaRP container (on port 8780/8781) - HaRP container -\> Nextcloud (through proxy or directly as the NC_INSTANCE_URL env var dictates) - ExApp -\> HaRP container (on port 8782) - ExApp -\> Nextcloud (through proxy or directly as the `Nextcloud URL` in the daemon config dictates) > **注意** > > The examples below use `some_very_secure_password` as the HaRP shared key. This is only a placeholder: generate your own random key for every installation, for example with `openssl rand -hex 32`. > > Use the same key for `HP_SHARED_KEY` of the HaRP container, the **HaRP shared key** field of the deploy daemon and, for a remote Docker host, `metadatas.token` in `frpc.toml`. Anyone who knows the key can talk to HaRP the way Nextcloud does, including managing ExApp containers on the Docker host, so keep it secret. ### Nextcloud and Docker on the same host - with Nextcloud bare metal The simplest configuration is when Nextcloud is installed on the host and docker is on the same host and applications are deployed to it. Create a HaRP container with either `--network host` option or expose the ports `8780` and `8782` to the host. ``` bash docker run \ -e HP_SHARED_KEY="some_very_secure_password" \ -e NC_INSTANCE_URL="https://127.0.0.1:8080" \ -v /var/run/docker.sock:/var/run/docker.sock \ -v `pwd`/certs:/certs \ --name appapi-harp -h appapi-harp \ --restart unless-stopped \ -p 8780:8780 \ -p 8782:8782 \ -d ghcr.io/nextcloud/nextcloud-appapi-harp:release ``` Go to AppAPI admin settings and register a `HaRP Proxy (Host)` daemon. ![AppAPI admin settings showing registration of HaRP Proxy (Host) daemon](https://docs.cocobook.com/uploads/images/gallery/2026-10/harp-host.png) Finally, test the whole setup with "Test deploy" in the 3-dots menu of the deploy daemon. ### Nextcloud and Docker on the same host - with Nextcloud in Docker When Nextcloud is installed in Docker, the HaRP container can be created in the same docker network as the Nextcloud instance. Create a HaRP container with `--network ` option, where `` is the name of the Docker network in which Nextcloud is accessible. ``` bash docker run \ -e HP_SHARED_KEY="some_very_secure_password" \ -e NC_INSTANCE_URL="https://nextcloud.tld" \ -v /var/run/docker.sock:/var/run/docker.sock \ -v `pwd`/certs:/certs \ --name appapi-harp -h appapi-harp \ --restart unless-stopped \ --net \ -d ghcr.io/nextcloud/nextcloud-appapi-harp:release ``` Go to AppAPI admin settings and register a `HaRP Proxy (Docker)` daemon. Take note of the `` value in the `Docker network` field. ![AppAPI admin settings showing registration of HaRP Proxy (Docker) daemon](https://docs.cocobook.com/uploads/images/gallery/2026-10/harp-docker.png) Finally, test the whole setup with "Test deploy" in the 3-dots menu of the deploy daemon. ### Docker on a remote host - with HaRP container on the local host This configuration is suited for deployments that want to offload the heavy lifting of the ExApps to a remote host, especially when using GPUs as compute devices. There can be multiple deploy daemons that can be used to deploy ExApps on different remote hosts for different compute capabilities. Here the HaRP container is deployed on the local host and the remote host tunnels the remote host's docker socket to the local host over the [FRP](https://github.com/fatedier/frp) secure tunnel. The ExApps are deployed on the remote host. A setup with the HaRP container itself on the remote is not supported. 1. Create a HaRP container in the local host following [the above examples](/link/986) but without the docker socket mount. > ``` bash > docker run \ > -e HP_SHARED_KEY="some_very_secure_password" \ > -e NC_INSTANCE_URL="https://127.0.0.1:8080" \ > -v `pwd`/certs:/certs \ > --name appapi-harp -h appapi-harp \ > --restart unless-stopped \ > -p 8780:8780 \ > -p 8782:8782 \ > -d ghcr.io/nextcloud/nextcloud-appapi-harp:release > ``` 2. Create a matching deploy daemon with `Docker socket proxy port` set to `24001`. > ![AppAPI deploy daemon configuration with Docker socket proxy port set to 24001](https://docs.cocobook.com/uploads/images/gallery/2026-10/harp-remote-24001.png) 3. The FRP generated client certificates should be present in the `certs` folder locally. Copy the files `client.crt`, `client.key` and `ca.crt` inside the `certs` folder to the remote host. 4. Create a folder structure on the remote host: `mkdir -p certs/frp` and copy the files `client.crt`, `client.key` and `ca.crt` to the `certs/frp` folder. 5. Create a new file `frpc.toml` with the following contents. > ``` toml > # frpc.toml > serverAddr = "your.harp.server.address" # Replace with your HP_FRP_ADDRESS host > serverPort = 8782 # Default port for FRP or the port your reverse proxy listens on > loginFailExit = false # If the FRP (HaRP) server is unavailable, continue trying to log in. > > transport.tls.certFile = "certs/frp/client.crt" > transport.tls.keyFile = "certs/frp/client.key" > transport.tls.trustedCaFile = "certs/frp/ca.crt" > transport.tls.serverName = "harp.nc" # DO NOT CHANGE THIS VALUE > > metadatas.token = "some_very_secure_password" # HP_SHARED_KEY in quotes > > [[proxies]] > remotePort = 24001 # Unique remotePort for each Docker Engine (range: 24001-24099) > name = "deploy-daemon-1" # Unique name for each Docker Engine > type = "tcp" > [proxies.plugin] > type = "unix_domain_socket" > unixPath = "/var/run/docker.sock" > ``` Make sure to replace the `your.harp.server.address` with the actual address of the local host where the HaRP container is running.\ You might want to open the port `8782` on the local host firewall to allow the remote host to connect to it,\ or use a reverse proxy to forward the requests to the HaRP container. An example with nginx is given below. Feel free to adjust the port you want to listen on. The FRP client will connect to this port exposed port.\ With the reverse proxy config below, the whole setup would only need the main Nextcloud proxy to be exposed and reachable from the outside world, simplifying the network setup. > ``` nginx > stream { > server { > listen 8782; # Replace with the port you want to listen on > proxy_pass 127.0.0.1:8782; > proxy_protocol off; > proxy_connect_timeout 10s; > proxy_timeout 300s; > } > } > ``` 6. Download a release of the FRP client from [the official releases](https://github.com/fatedier/frp/releases/latest) or [our snapshot from here](https://github.com/nextcloud/HaRP/tree/main/exapps_dev). 7. Extract and copy the `frpc` binary to an appropriate location on the remote host, e.g. `/usr/local/bin`. 8. Make it executable: `chmod +x /usr/local/bin/frpc`. 9. Start the FRP client with the command: `frpc -c /path/to/frpc.toml`. 10. Finally, test the whole setup with "Test deploy" in the 3-dots menu of the deploy daemon. ### Docker / Reverse Proxy / Nextcloud on 3 independent hosts - with HaRP container This is the related infrastructure ``` mermaid stateDiagram-v2 classDef docker fill: #1f97ee, color: white, font-size: 34px, stroke: #364c53, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/docker.png) no-repeat center center / contain classDef nextcloud fill: #006aa3, color: white, font-size: 34px, stroke: #045987, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/nextcloud-svg.png) no-repeat center center / contain classDef python fill: #1e415f, color: white, stroke: #364c53, stroke-width: 1px Direction LR Host1 --> Host2 : by port Host3 --> Host1 : by port Host3 --> Host2 : by port state Host1 { Nextcloud } state Host2 { [*] --> HaRP : by port Daemon --> Containers state Containers { [*] --> HaRP : /var/run/docker.sock HaRP --> ExApp1 HaRP --> ExApp2 HaRP --> ExApp3 } } state Host3 { Apache Reverse Proxy } class Nextcloud nextcloud class Daemon docker class ExApp1 python class ExApp2 python class ExApp3 python ``` Please see below the steps I follow All of the following steps are based on a Almalinux Distro. Please customize for your distribution. 1. On the Host2 Docker 1.1. Creation of Cert folder (if necessary) > ``` bash > mkdir -p /some/path/certs > ``` 1.2. Open ports > ``` bash > firewall-cmd --permanent --zone=public --add-port=8780/tcp > firewall-cmd --permanent --zone=public --add-port=8782/tcp > firewall-cmd --reload > ``` 1.3. Deploy of the HaRP Container > ``` bash > docker run \ > -e HP_SHARED_KEY="some_very_secure_password" \ > -e NC_INSTANCE_URL="https://cloud.acme.com" \ > -e HP_TRUSTED_PROXY_IPS="192.168.0.0/24" \ > -v /var/run/docker.sock:/var/run/docker.sock \ > -v /some/path/certs:/certs \ > -p 8780:8780 \ > -p 8782:8782 \ > --name appapi-harp -h appapi-harp \ > --restart unless-stopped \ > -d ghcr.io/nextcloud/nextcloud-appapi-harp:release > ``` > > Replace `192.168.0.0/24` with the IP address or subnet of your reverse proxy (Host3). 2. On the Host3 Apache Reverse Proxy - Reverse proxy redirections On the virtual Host "cloud.acme.com" of the apache conf file Add the following lines (before the existing configuration) > ``` apache > # AppAPI Configuration > ProxyPass /exapps/ http://:8780/exapps/ > ProxyPassReverse /exapps/ http://:8780/exapps/ > ``` 3. On the Nextcloud Web Interface - Daemon Register Add the following configuration : - Daemon configuration template : `HaRP Proxy (HOST)` - Surname : `appapi-harp` - Display name : `appapi-harp` - Deployment method : `docker-install` - HaRP host : `:8780` - HaRP shared key : `some_very_secure_password` - Nextcloud URL : `https://cloud.acme.com` - FRP server address : `:8782` - Docker network : `bridge` Finally, test the whole setup with “Test deploy” in the 3-dots menu of the deploy daemon. 4. Additional tests from the network of your hosts > **提示** > > The `docker-engine-port` header tells HaRP which internal virtual port to route to the Docker engine. When HaRP has the Docker socket mounted directly (as in this setup), it uses `24000` as the default virtual port for that local socket. This value does not require any additional firewall or port configuration. > > ``` bash > ``` > > curl -fsS -H "harp-shared-key: some_very_secure_password" -H "docker-engine-port: 24000" \:8780/exapps/app_api/v1.41/ping > > ``` bash > ``` > > curl -fsS -H "harp-shared-key: some_very_secure_password" -H "docker-engine-port: 24000" ## Docker Deploy Daemon (Docker Socket Proxy) ### NC & Docker on the Same-Host The simplest configuration is when Nextcloud is installed on the host and Docker is on the same host and applications are deployed to it. ``` mermaid stateDiagram-v2 classDef docker fill: #1f97ee, color: white, font-size: 34px, stroke: #364c53, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/docker.png) no-repeat center center / contain classDef nextcloud fill: #006aa3, color: white, font-size: 34px, stroke: #045987, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/nextcloud-svg.png) no-repeat center center / contain classDef python fill: #1e415f, color: white, stroke: #364c53, stroke-width: 1px Host state Host { Nextcloud --> Daemon : /var/run/docker.sock Daemon --> Containers state Containers { ExApp1 -- ExApp2 -- ExApp3 } } class Nextcloud nextcloud class Daemon docker class ExApp1 python class ExApp2 python class ExApp3 python ``` Suggested config values(template *Custom default*): 1. Daemon host: `/var/run/docker.sock` 2. HTTPS checkbox: *not supported using docker socket* 3. Network: `host` 4. HaProxy password: **not supported using raw docker socket, should be empty** --- Suggested way to communicate with Docker via [Docker Socket Proxy container](https://github.com/nextcloud/docker-socket-proxy). ``` mermaid stateDiagram-v2 classDef docker fill: #1f97ee, color: white, font-size: 34px, stroke: #364c53, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/docker.png) no-repeat center center / contain classDef nextcloud fill: #006aa3, color: white, font-size: 34px, stroke: #045987, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/nextcloud-svg.png) no-repeat center center / contain classDef python fill: #1e415f, color: white, stroke: #364c53, stroke-width: 1px Host state Host { Nextcloud --> DockerSocketProxy: by port Docker --> Containers Docker --> DockerSocketProxy : /var/run/docker.sock state Containers { DockerSocketProxy --> ExApp1 DockerSocketProxy --> ExApp2 DockerSocketProxy --> ExApp3 } } class Nextcloud nextcloud class Docker docker class ExApp1 python class ExApp2 python class ExApp3 python ``` Suggested config values(template *Docker Socket Proxy*): 1. Daemon host: `localhost:2375` Choose **A** or **B** option: 1. Docker Socket Proxy should be deployed with `network=host` and `BIND_ADDRESS=127.0.0.1` 2. Docker Socket Proxy should be deployed with `network=bridge` and it's port should be published to host's 127.0.0.1(e.g. **-p 127.0.0.1:2375:2375**) 2. HTTPS checkbox: **disabled** 3. Network: `host` 4. HaProxy password: **should not be empty** > **注意** > > Be careful with option `A`, by default **Docker Socket Proxy** binds to `*` if `BIND_ADDRESS` is not specified during container creation. Check opened ports after finishing configuration. ### Docker on a remote host Distributed configuration occurs when Nextcloud is installed on one host and Docker is located on a remote host, resulting in the deployment of applications on the remote host. Benefit: no performance impact on Nextcloud host. In this case, the AppAPI uses a Docker Socket Proxy deployed on remote host to access docker socket and ExApps. ``` mermaid stateDiagram-v2 classDef docker fill: #1f97ee, color: white, font-size: 34px, stroke: #364c53, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/docker.png) no-repeat center center / contain classDef nextcloud fill: #006aa3, color: white, font-size: 34px, stroke: #045987, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/nextcloud-svg.png) no-repeat center center / contain classDef python fill: #1e415f, color: white, stroke: #364c53, stroke-width: 1px Direction LR Host1 --> Host2 : by port state Host1 { Nextcloud } state Host2 { [*] --> DockerSocketProxy : by port Daemon --> Containers state Containers { [*] --> DockerSocketProxy : /var/run/docker.sock DockerSocketProxy --> ExApp1 DockerSocketProxy --> ExApp2 DockerSocketProxy --> ExApp3 } } class Nextcloud nextcloud class Daemon docker class ExApp1 python class ExApp2 python class ExApp3 python ``` Suggested config values(template *Docker Socket Proxy*): 1. Daemon host: ADDRESS_OF_REMOTE_MACHINE (e.g. **server_name.com:2375**) 2. HTTPS checkbox: `enabled` 3. Network: `host` 4. HaProxy password: **should not be empty** ### NC & ExApps in the same Docker Applications are deployed in the same Docker where Nextcloud resides. Suggested way to communicate with Docker: via `docker-socket-proxy`. ``` mermaid stateDiagram-v2 classDef docker fill: #1f97ee, color: white, font-size: 34px, stroke: #364c53, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/docker.png) no-repeat center center / contain classDef nextcloud fill: #006aa3, color: white, font-size: 34px, stroke: #045987, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/nextcloud-svg.png) no-repeat center center / contain classDef python fill: #1e415f, color: white, stroke: #364c53, stroke-width: 1px Host state Host { Daemon --> Containers state Containers { [*] --> DockerSocketProxy : /var/run/docker.sock Nextcloud --> DockerSocketProxy: by port -- DockerSocketProxy --> ExApp1 DockerSocketProxy --> ExApp2 } } class Nextcloud nextcloud class Daemon docker class ExApp1 python class ExApp2 python class ExApp3 python ``` Suggested config values(template *Docker Socket Proxy*): 1. Daemon host: nextcloud-appapi-dsp:2375 2. HTTPS checkbox: `disabled` 3. Network: [user defined network](https://docs.docker.com/network/#user-defined-networks) 4. HaProxy password: **should not be empty** > **提示** > > Network **should not be the default docker's bridge** as it does not support DNS resolving by container names. > > This means that **Docker Socket Proxy**, **Nextcloud** and **ExApps** containers should all be in the same docker network, different from the default **bridge**. ### Nextcloud in Docker AIO (all-in-one) In the case of AppAPI in Docker AIO setup (installed in Nextcloud container). > **提示** > > AIO Docker Socket Proxy container must be enabled. ``` mermaid stateDiagram-v2 classDef docker fill: #1f97ee, color: white, font-size: 34px, stroke: #364c53, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/docker.png) no-repeat center center / contain classDef docker2 fill: #1f97ee, color: white, font-size: 20px, stroke: #364c53, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/docker.png) no-repeat center center / contain classDef nextcloud fill: #006aa3, color: white, font-size: 34px, stroke: #045987, stroke-width: 1px, background: url(https://docs.cocobook.com/uploads/images/gallery/2026-10/nextcloud-svg.png) no-repeat center center / contain classDef python fill: #1e415f, color: white, stroke: #364c53, stroke-width: 1px Host state Host { Daemon --> Containers state Containers { [*] --> NextcloudAIOMasterContainer : /var/run/docker.sock [*] --> DockerSocketProxy : /var/run/docker.sock NextcloudAIOMasterContainer --> Nextcloud AppAPI --> Nextcloud : installed in Nextcloud --> DockerSocketProxy DockerSocketProxy --> ExApp1 DockerSocketProxy --> ExApp2 DockerSocketProxy --> ExApp3 } } class Nextcloud nextcloud class Daemon docker class Daemon2 docker2 class ExApp1 python class ExApp2 python class ExApp3 python ``` AppAPI will automatically create the default DaemonConfig for AIO Docker Socket Proxy in order to use it as an orchestrator to create ExApp containers. > **提示** > > Default DaemonConfig will be created only if the default DaemonConfig is not already registered. #### Default AIO Deploy Daemon (Docker Socket Proxy) Nextcloud AIO has a specifically created Docker Socket Proxy container to be used as the Deploy Daemon in AppAPI. It has [fixed parameters](https://github.com/nextcloud/app_api/blob/main/lib/DeployActions/AIODockerActions.php#L52-L74)): - Name: `docker_aio` - Display name: `AIO Docker Socket Proxy` - Accepts Deploy ID: `docker-install` - Protocol: `http` - Host: `nextcloud-aio-docker-socket-proxy:2375` - Compute device: `CPU` - Network: `nextcloud-aio` - Nextcloud URL (passed to ExApps): `https://$NC_DOMAIN` #### Docker Socket Proxy security AIO Docker Socket Proxy has strictly limited access to the Docker APIs described in [HAProxy configuration](https://github.com/nextcloud/all-in-one/blob/main/Containers/docker-socket-proxy/haproxy.cfg). ## Container log rotation AppAPI deploys each ExApp as a container but does not set a logging configuration on it, so each ExApp container uses the logging settings of the deploy daemon that runs it. If those logs have no size limit they are never rotated and can grow until they fill the host disk. Configure rotation on the daemon, where it applies to every container the daemon runs. ### Docker Docker's default `json-file` driver sets no size limit. Add the following to the Docker daemon configuration file (`/etc/docker/daemon.json` on most Linux hosts) and restart the daemon: ``` json { "log-driver": "json-file", "log-opts": { "max-size": "10m", "max-file": "3" } } ``` ``` bash sudo systemctl restart docker ``` Each container then keeps at most `max-file` log files of `max-size` each (here three files of 10 MB, so up to 30 MB per container). Adjust the values to suit your hosts. > **提示** > > A container keeps the logging configuration it was created with, and disabling and enabling an ExApp only stops and starts the existing container. Remove and install the affected ExApps again (or update them) so their containers are recreated with the new settings. > **说明** > > Keep the `json-file` or `journald` driver: AppAPI can only download container logs for these two drivers, so a different driver disables log downloads for ExApp containers. If you deliberately use a different driver (`syslog`, `fluentd`, and so on) to forward logs elsewhere, keep your existing setup: rotation is then handled by that driver and the steps above do not apply. ### Podman When the deploy daemon is Podman, set a global default in `/etc/containers/containers.conf` (or the per-user file) and re-deploy the affected ExApps: ``` toml [containers] log_driver = "json-file" log_size_max = 10485760 ``` ### Kubernetes For the Kubernetes deploy daemon, container log rotation is handled by the kubelet, not by AppAPI. Set `containerLogMaxSize` and `containerLogMaxFiles` in the kubelet configuration on each node and restart the kubelet for the change to take effect (it loads its configuration only at startup); see the [Kubernetes logging architecture](https://kubernetes.io/docs/concepts/cluster-administration/logging/) documentation. ## NC to ExApp Communication Communications between Nextcloud and ExApps are done via the AppAPI. With Docker Socket Proxy, the requests are sent to the ExApp container directly. For HaRP, the communication goes through the main Nextcloud proxy and the HaRP container. Each type of DeployDaemon necessarily implements the `resolveExAppUrl` function. It has the prototype: ``` php public function resolveExAppUrl( string $appId, string $protocol, string $host, array $deployConfig, int $port, array &$auth ) {} ``` where: - **protocol** is daemon protocol value - **host** is daemon host value, *can be DNS:port or IP:PORT or even path to docker socket*. - **port** is an integer with ExApp port - **deployConfig** can be custom for each Daemon type - **auth** is an optional array, with *Basic Authentication* data if needed to access ExApp > **提示** > > Applies only to Docker Socket Proxy. > > The optional additional parameter *OVERRIDE_APP_HOST* can be used to override the host that will be used for ExApp binding. > > It can be `0.0.0.0` in some specific configurations, when VPN is used or both Nextcloud instance and ExApps are one the same physical machine but different virtual environments. > > Also you can specify something like `10.10.2.5` and in this case `ExApp` will try to bind to that address and AppAPI will try to send request s directly to this address assuming that ExApp itself bound on it. The simplest implementation is in the **Manual-Install** deploy type: ``` php public function resolveExAppUrl( string $appId, string $protocol, string $host, array $deployConfig, int $port, array &$auth ): string { if (boolval($deployConfig['harp'] ?? false)) { $url = rtrim($deployConfig['nextcloud_url'], '/'); if (str_ends_with($url, '/index.php')) { $url = substr($url, 0, -10); } return sprintf('%s/exapps/%s', $url, $appId); } $auth = []; if (isset($deployConfig['additional_options']['OVERRIDE_APP_HOST']) && $deployConfig['additional_options']['OVERRIDE_APP_HOST'] !== '' ) { $wideNetworkAddresses = ['0.0.0.0', '127.0.0.1', '::', '::1']; if (!in_array($deployConfig['additional_options']['OVERRIDE_APP_HOST'], $wideNetworkAddresses)) { $host = $deployConfig['additional_options']['OVERRIDE_APP_HOST']; } } return sprintf('%s://%s:%s', $protocol, $host, $port); } ``` Here we see that AppAPI sends requests to the **host**:**port** specified during daemon creation for manual-install without HaRP.\ But it exclusively uses the `http(s)://nextcloud.example.tld/exapps/` route for manual deployments using the HaRP proxy. `http(s)://nextcloud.example.tld` is the Nextcloud URL specified in the daemon config. Take care to configure the `/exapps/` route in your reverse proxy accordingly if your Nextcloud instance is on a subpath `https://nextcloud.example.tld/nextcloud`. See [Configuring Your Reverse Proxy](https://github.com/nextcloud/harp?tab=readme-ov-file#configuring-your-reverse-proxy) in the HaRP readme for examples. Now, let's take a look at the Docker Daemon implementation of `resolveExAppUrl`: ``` php public function resolveExAppUrl( string $appId, string $protocol, string $host, array $deployConfig, int $port, array &$auth ): string { if (boolval($deployConfig['harp'] ?? false)) { $url = rtrim($deployConfig['nextcloud_url'], '/'); if (str_ends_with($url, '/index.php')) { $url = substr($url, 0, -10); } return sprintf('%s/exapps/%s', $url, $appId); } $auth = []; if (isset($deployConfig['additional_options']['OVERRIDE_APP_HOST']) && $deployConfig['additional_options']['OVERRIDE_APP_HOST'] !== '' ) { $wideNetworkAddresses = ['0.0.0.0', '127.0.0.1', '::', '::1']; if (!in_array($deployConfig['additional_options']['OVERRIDE_APP_HOST'], $wideNetworkAddresses)) { return sprintf( '%s://%s:%s', $protocol, $deployConfig['additional_options']['OVERRIDE_APP_HOST'], $port ); } } $host = explode(':', $host)[0]; if ($protocol == 'https') { $exAppHost = $host; } elseif (isset($deployConfig['net']) && $deployConfig['net'] === 'host') { $exAppHost = 'localhost'; } else { $exAppHost = $appId; } if ($protocol == 'https' && isset($deployConfig['haproxy_password']) && $deployConfig['haproxy_password'] !== '') { // we only set haproxy auth for remote installations, when all requests come through HaProxy. $haproxyPass = $this->crypto->decrypt($deployConfig['haproxy_password']); $auth = [self::APP_API_HAPROXY_USER, $haproxyPass]; } return sprintf('%s://%s:%s', $protocol, $exAppHost, $port); } ``` The route for HaRP setups remain the same here as in the previous example. All the requests are sent to the Nextcloud URL with the `/exapps/` route. For Docker Socket Proxy, however, we have much more complex algorithm of detecting to where requests should be send. First of all, if the protocol is set to `https`, AppAPI always sends requests to the daemon host, and in this case, it is a HaProxy that will forward requests to ExApps that will be listening on `localhost`. Briefly, it will look like this (*haproxy_host==daemon host value*): NC --\> *https* --\> `haproxy_host:ex_app_port` --\> *http* --\> `localhost:ex_app_port` When the protocol is not `https` but `http`, then what will be the endpoint where to send requests is determined by `$deployConfig['net']` value. If `net` is defined and equal to `host`, then AppAPI assumes that ExApp is installed somewhere in the current host network and will be available on `localhost` loop-back adapter. NC --\> *http* --\> `localhost:ex_app_port` In all other cases, the ExApp should be available by it's name: e.g. when using docker **custom bridge** network all containers available by DNS. NC --\> *http* --\> `app_container_name:ex_app_port` These three different types of communication cover most popular configurations. # Managing Deploy Daemons > # Managing Deploy Daemons ## OCC CLI There are a few OCC CLI commands to manage Deploy Daemons: 1. Register `occ app_api:daemon:register` 2. Unregister `occ app_api:daemon:unregister` 3. List registered daemons `occ app_api:daemon:list` ### Register Register Deploy Daemon (DaemonConfig). Command: `app_api:daemon:register [--net NET] [--haproxy_password HAPROXY_PASSWORD] [--compute_device COMPUTE_DEVICE] [--set-default] [--harp] [--harp_frp_address HARP_FRP_ADDRESS] [--harp_shared_key HARP_SHARED_KEY] [--harp_docker_socket_port HARP_DOCKER_SOCKET_PORT] [--harp_exapp_direct] [--] ` #### Arguments - `name` - unique name of the daemon (e.g. `docker_local_sock`) - `display-name` - name of the daemon (e.g. `My Local Docker`, will be displayed in the UI) - `accepts-deploy-id` - type of deployment (`docker-install` or `manual-install`) - `host` - **path to docker-socket** or the Docker Socket Proxy: `address:port` - `protocol` - protocol used to communicate with the Daemon/ExApps (`http` or `https`) - `nextcloud_url` - Nextcloud URL, Daemon config required option (e.g. `https://nextcloud.local`) #### Options - `--net [network-name]` - `[required]` network name to bind docker container to (default: `host`) - `--haproxy_password HAPROXY_PASSWORD` - `[optional]` password for AppAPI Docker Socket Proxy - `--compute_device GPU` - `[optional]` GPU device to expose to the daemon (e.g. `cpu|cuda|rocm`, default: `cpu`) - `--set-default` - `[optional]` set created daemon as default for ExApps installation - `--harp` - `[optional]` Flag to set daemon to use HaRP for all docker and exapp communication - `--harp_frp_address` - `[optional]` \[host\]:\[port\] of the HaRP FRP server, default host is same as HaRP host and port is 8782 - `--harp_shared_key` - `[optional]` HaRP shared key for secure communication between HaRP and AppAPI. It must match `HP_SHARED_KEY` of the HaRP container. The `some_very_secure_password` value in the examples below is only a placeholder, see [HaRP shared key](/link/986). - `--harp_docker_socket_port` - `[optional]` 'remotePort' of the FRP client of the remote docker socket proxy. There is one included in the harp container so this can be skipped for default setups. (default: "24000") - `--harp_exapp_direct` - `[optional]` Flag for the advanced setups only. Disables the FRP tunnel between ExApps and HaRP. #### Usage Examples - Register a HaRP deploy daemon within the `nextcloud` docker network, with the `appapi-harp` container as the host and the `appapi-harp:8782` as the FRP server address. This can be paired with a HaRP container running in the same network. > ``` bash > occ app_api:daemon:register harp_proxy_docker "Harp Proxy (Docker)" "docker-install" "http" "appapi-harp:8780" "http://nextcloud.local" --net nextcloud --harp --harp_frp_address "appapi-harp:8782" --harp_shared_key "some_very_secure_password" --set-default --compute_device=cuda > ``` - Register a HaRP deploy daemon with the `localhost` as the host and the `localhost:8782` as the FRP server address. This can be paired with a HaRP container running in the host network mode or that has exposed the ports `8780` and `8782` to the host. > ``` bash > app_api:daemon:register harp_proxy_host "Harp Proxy (Host)" "docker-install" "http" "localhost:8780" "http://nextcloud.local" --harp --harp_frp_address "localhost:8782" --harp_shared_key "some_very_secure_password" --set-default --compute_device=cuda > ``` - Register a manual install deploy daemon with HaRP support. This can be paired with a HaRP container running in the same network. The HaRP container need not have access to a docker socket or any other ports exposed to the host. It will not create docker containers of the ExApps but will only proxy the requests to the ExApp process manually launched by the user. > > **提示** > > > > \| The ExApp process should have a FRP Client (frpc) running in the same network as the HaRP container or should be able to connect to the ports exposed by the HaRP container. \| If the communication has to go without the FRP client, the `--harp_exapp_direct` flag should be provided. The localhost IP address is always used as the host in this case for manual deployments and `OVERRIDE_APP_HOST` or the `` is used for ExApp deployments. Take care not to use the host network mode or the default bridge network for this. > > ``` bash > app_api:daemon:register manual_install_harp "Harp Manual Install" "manual-install" "http" "appapi-harp:8780" "http://nextcloud.local" --net nextcloud --harp --harp_frp_address "appapi-harp:8782" --harp_shared_key "some_very_secure_password" > ``` - Register a Docker Socket Proxy deploy daemon with the `nextcloud-appapi-dsp:2375` as the host and the `nextcloud` docker network. This can be paired with a Docker Socket Proxy container running in the same network with the default port `2375`. > ``` bash > app_api:daemon:register docker_install "Docker Socket Proxy" "docker-install" "http" "nextcloud-appapi-dsp:2375" "http://nextcloud.local" --net=nextcloud --set-default --compute_device=cuda > ``` - Register a manual deploy daemon with `host.docker.internal` as the host used to connect to the ExApps. > ``` bash > app_api:daemon:register manual_install "Manual Install" "manual-install" "http" null "http://nextcloud.local" > ``` - Register a local docker deploy daemon with the `/var/run/docker.sock` as the socket and the host, and the `nextcloud` docker network. This does not need a Docker Socket Proxy container. The compute device used by this daemon is `CPU`. > ``` bash > app_api:daemon:register local_docker "Docker Local" "docker-install" "http" "/var/run/docker.sock" "http://nextcloud.local" --net=nextcloud > ``` - Register a local docker deploy daemon with the `/var/run/docker.sock` as the socket and the host, and the `nextcloud` docker network. This does not need a Docker Socket Proxy container. The compute device used by this daemon is `CUDA` (NVIDIA). > ``` bash > app_api:daemon:register local_docker "Docker Local" "docker-install" "http" "/var/run/docker.sock" "http://nextcloud.local" --net=nextcloud --set-default --compute_device=cuda > ``` #### DeployConfig DeployConfig is a set of additional options in Daemon config, which are used in deployment algorithms to configure ExApp container. ``` json { "net": "host", "nextcloud_url": "https://nextcloud.local", "haproxy_password": "some_secure_password", "computeDevice": { "id": "cuda", "name": "CUDA (NVIDIA)", }, "harp": { "frp_address": "localhost:8782", "docker_socket_port": "24000", "exapp_direct": false } } ``` #### DeployConfig options > - `net` **\[required\]** - network name to bind docker container to (default: `host`) > > - `nextcloud_url` **\[required\]** - Nextcloud URL (e.g. `https://nextcloud.local`) > > - `haproxy_password` *\[optional\]* - password for AppAPI Docker Socket Proxy > > - `computeDevice` *\[optional\]* - Compute device to attach to the daemon (e.g. `{ "id": "cuda", "label": "CUDA (NVIDIA)" }`) > > - `harp` *\[optional\]* - HaRP options, can be `null` in case of non-HaRP setups > - `frp_address` *\[optional\]* - \[host\]:\[port\] of the HaRP FRP server, default host is same as HaRP host and port is 8782 > - `docker_socket_port` *\[optional\]* - 'remotePort' of the FRP client of the remote docker socket proxy. There is one included in the harp container so this can be skipped for default setups. \[default: "24000"\] > - `exapp_direct` *\[optional\]* - Flag for the advanced setups only. Disables the FRP tunnel between ExApps and HaRP. ### Unregister Unregister Deploy Daemon (DaemonConfig). Command: `app_api:daemon:unregister ` ### List registered daemons List registered Deploy Daemons (DaemonConfigs). Command: `app_api:daemon:list` ## Nextcloud AIO In the case of AppAPI installed in AIO, a default Deploy Daemon is registered automatically. It is possible to register additional Deploy Daemons using the same methods as described above. ## Additional options There is a possibility to add additional options to the Deploy Daemon configuration, which are key-value pairs.\ This should not be used for HaRP. Currently, the following options are available: > - `OVERRIDE_APP_HOST` - can be used to override the host that will be used for ExApp binding (not passed to ExApp container envs) # Test Deploy Daemon You can test each Daemon configuration deployment from the AppAPI Admin settings. ![AppAPI admin settings showing daemon configuration with Test deploy option in three-dots menu](https://docs.cocobook.com/uploads/images/gallery/2026-10/test-deploy.png) ## Status Checks The Deploy test installs a [test-deploy](https://github.com/nextcloud/test-deploy) ExApp to verify each step of the deployment process, including a hardware support check - for each compute device, there is a separate Docker image. > **提示** > > The Test Deploy ExApp container is not removed after the test as it's needed for logs and status checks. You can remove it after testing from the Apps page. The Docker images are also not removed from the Daemon; you can clean up unused images with the `docker image prune` command. ![AppAPI test deploy modal showing deployment status checks](https://docs.cocobook.com/uploads/images/gallery/2026-10/test-deploy-modal-4.png) ### Register The Register step is the first step; it checks if the ExApp is registered in Nextcloud. ### Image Pull The Image Pull step downloads the ExApp Docker image. Possible errors: - Image not found (e.g. not public, no image found for your hardware architecture) - Image pull failed (e.g., due to network issues) - Image pull timeout - Your Docker Socket Proxy/HaRP is not configured correctly and blocks access to this Docker Engine API See `journalctl -f -u docker.service` for more details in systemd based systems. ### Container Started The Container Started step verifies that the ExApp container is created and started successfully. Possible errors: - Container failed to start with GPU support (may be missing or misconfigured) - For NVIDIA, refer to the [NVIDIA Docker configuration docs](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). - For AMD, refer to the [ROCm Docker configuration docs](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html). - The ExApp issue during startup (e.g. not enough memory). The app would show repeated starting attempts in the logs. See `docker logs nc_app_` for more details. ### Heartbeat The Heartbeat step checks if the container's health check is finished and the container is healthy. The ExApp might have additional pre-configuration logic during this step. Possible errors: - ExApp failed to start a web server, e.g., if the port is already in use (this should be visible in the container logs) - ExApp heartbeat_count keeps increasing, this may indicate that the ExApp couldn't start properly - Nextcloud can not reach the ExApp container, e.g., - due to a network issue or a firewall (this should be visible in the server logs or the firewall logs) - due to a "http" protocol deploy daemon. In this case, the ExApp's container listens on localhost (127.0.0.1 or ::1) which might not be reachable from the Nextcloud server and you might want to listen on a different IP address. See `OVERRIDE_APP_HOST` in [Additional options](/link/987) in the Deploy Daemon form. This issue can be identified using this command: `lsof -i -P -n | grep LISTEN` - For HaRP, the main Nextcloud proxy might not be configured to redirect requests to the HaRP container correctly. See the [Configuring Your Reverse Proxy](https://github.com/nextcloud/harp?tab=readme-ov-file#configuring-your-reverse-proxy) section in the HaRP readme. ### Init The Init step checks if the ExApp is initialized and ready to use. During the init step, the ExApp may perform downloads of extra stuff required for it. Possible errors: - Initialization failed (e.g., due to network issues or timeout) - ExApp not being able to reach the Nextcloud server (e.g., due to a network issue or a firewall) ### Enabled The Enabled step checks if the ExApp is enabled and ready to use. During this step, the ExApp registers all the required and available APIs of the Nextcloud AppFramework. Possible errors: - ExApp did not respond to the enable request - ExApp failed to enable due to a failure in registering AppAPI Nextcloud AppFramework APIs (this should be visible both in the container logs and in the Nextcloud logs if there are any errors) ## Download Logs You can download the logs of the last test deploy attempt container. > **提示** > > Downloading Docker container logs is only possible for containers using the json-file or journald logging drivers. # Managing ExApps Managing ExApps can be done from App Management UI as with other Nextcloud Apps, but you can also use the AppAPI commands in the OCC CLI tool. There are several commands to work with ExApps: 1. Register 2. Unregister 3. Update 4. Enable 5. Disable 6. List ExApps ## Register Command: `app_api:app:register [--force-scopes] [--info-xml INFO-XML] [--json-info JSON-INFO] [--wait-finish] [--silent] [--test-deploy-mode] [--env [ENV]] [--mount [MOUNT]] [--] []` The register command is the first ExApp installation step. ### Arguments > - `appid` - unique name of the ExApp (e.g. `app_python_skeleton`, must be the same as in deployed container) > - `daemon-config-name` - unique name of the daemon (e.g. `docker_local_sock`) ### Options > - `--json-info JSON-INFO` *\[optional\]* - ExApp deploy JSON info (json string) > - `--info-xml INFO-XML` *\[optional\]* - path to info.xml file (url or local absolute path) > - `--wait-finish` *\[optional\]* - wait until initialization finished > - `--silent` *\[optional\]* - do not print to console > - `--test-deploy-mode` *\[optional\]* - test deploy mode with additional status checks and slightly different logic ### Advanced Deploy Options > - `--env` *\[optional\]* - environment (ENV_NAME=ENV_VALUE), passed to ExApp container as environment variables (multiple values allowed) > - `--mount` *\[optional\]* - mount options (SRC_PATH=DST_PATH), passed to ExApp container as volume mounts (multiple values allowed) ## Unregister Command: `app_api:app:unregister [--rm-data] [--force] [--silent] [--] ` To remove an ExApp, you can use the unregister command. By default, this command does *not* delete the ExApp's persistent storage (data volume) to avoid accidental removal of any user data. ### Arguments > - `appid` - unique name of the ExApp (e.g. `app_python_skeleton`, must be the same as in deployed container) ### Options > - `--rm-data` *\[optional\]* - remove ExApp persistent storage (data volume) > - `--force` *\[optional\]* - continue removal even if some error occurs > - `--silent` *\[optional\]* - print a minimum of information, display only some errors, if any ## Update Command: `app_api:app:update [--info-xml INFO-XML] [--force-update] [-e|--enabled] [--] ` ExApp will be updated if there is a new version available. ### Arguments > - `appid` - unique name of the ExApp (e.g. `app_python_skeleton`, must be the same as in deployed container) ### Options > - `--info-xml INFO-XML` *\[optional\]* - path to info.xml file (url or local absolute path) > - `-e|--enabled` *\[optional\]* - enable ExApp after update ## Enable Command: `app_api:app:enable ` ## Disable Command: `app_api:app:disable ` ## List ExApps Command: `app_api:app:list` ListExApps command will show all ExApps: ExApps: appid (Display Name): version [enabled/disabled] to_gif_example (To Gif Example): 1.0.0 [enabled] upscaler_example (Upscaler Example): 1.0.0 [enabled] # Advanced Deploy Options AppAPI allows optionally to configure environment variables and mounts for the ExApp container. It is available via "Deploy options" modal next to "Deploy and Enable" button in the sidebar of the ExApp page on the Apps management page: ![AppAPI apps page showing the Deploy options button next to Deploy and Enable in the ExApp sidebar](https://docs.cocobook.com/uploads/images/gallery/2026-10/advanced-deploy-options-1.png) Or via CLI (advanced_deploy_options_cli). ## Environment Variables Environment variables enable more precise configuration of the ExApp. ExApp developers can define the list of supported environment variables with descriptions, only these variables will be available for configuration. By default there are only mounts available for configuration. ![AppAPI Deploy options modal showing environment variables configuration](https://docs.cocobook.com/uploads/images/gallery/2026-10/advanced-deploy-options-2.png) When ExApp installed the list of set environment variables will be displayed. ## Mounts Mounts can be used to provide additional data to the ExApp container from the host. For example, it will be useful for some apps to provide a folder with SSL certs of your cloud, so the app can handle HTTPS correctly without any additional re-installation of the ExApp. ![AppAPI Deploy options modal showing mounts configuration](https://docs.cocobook.com/uploads/images/gallery/2026-10/advanced-deploy-options-3.png) # Artificial Intelligence - [Overview](/link/992) - [Nextcloud Assistant](/link/993) - [App: Local Machine translation 2 (translate2)](/link/994) - [App: Local large language model (llm2)](/link/995) - [App: Local Whisper Speech-To-Text (stt_whisper2)](/link/996) - [App: Local Image Generation (text2image_stablediffusion2)](/link/997) - [App: Recognize](/link/998) - [App: Context Chat](/link/999) - [App: Context Agent (context_agent)](/link/1000) - [App: Summary Bot (Talk chat summarize bot)](/link/1001) - [App: Local Text-To-Speech (text2speech_kokoro)](/link/1002) - [App: Live Transcription and Translation in Nextcloud Talk (live_transcription)](/link/1003) - [App: Local Optical Character Recognition (ocr_paddle)](/link/1004) - [AI as a Service](/link/1005) - [Insight and debugging](/link/1006) - [Legal: Compliance with EU AI Act](/link/1007) # Overview We strive to bring Artificial Intelligence features to Nextcloud. This section highlights these features, how they work and where to find them. All of these features are completely optional. If you want to have them on your server, you need to install them via separate Nextcloud Apps. ## Overview of AI features Nextcloud uses modularity to separate raw AI functionality from the Graphical User interfaces and apps that make use of said functionality. Each instance can thus make use of various backends that provide the functionality for the same frontends and the same functionality can be implemented by multiple apps using on-premises processing or third-party AI service providers. ![](https://docs.cocobook.com/uploads/images/gallery/2026-10/ai-overview.png) | Feature | App | Rating | Open source | Freely available model | Freely available training data | Privacy: Keeps data on premises | |----|----|----|----|----|----|----| | Smart inbox | [Mail](https://apps.nextcloud.com/apps/mail) | Green | Yes | Yes | Yes | Yes | | Image object recognition | [Recognize](https://apps.nextcloud.com/apps/recognize) | Green | Yes | Yes | Yes | Yes | | Image face recognition | [Recognize](https://apps.nextcloud.com/apps/recognize) | Green | Yes | Yes | Yes | Yes | | Video action recognition | [Recognize](https://apps.nextcloud.com/apps/recognize) | Green | Yes | Yes | Yes | Yes | | Audio music genre recognition | [Recognize](https://apps.nextcloud.com/apps/recognize) | Green | Yes | Yes | Yes | Yes | | Suspicious login detection | [Suspicious Login](https://apps.nextcloud.com/apps/suspicious_login) | Green | Yes | Yes | Yes | Yes | | Related resources | [Related Resources](https://apps.nextcloud.com/apps/related_resources) | Green | Yes | Yes | Yes | Yes | | Recommended files | recommended_files | Green | Yes | Yes | Yes | Yes | | Text processing using LLMs | [Local large language model 2 (ExApp) (ExApp)](https://apps.nextcloud.com/apps/llm2) | Green | Yes | Yes - Llama 3.1 model by Meta | Yes | Yes | | | [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) | Red | No | No | No | No | | | [OpenAI and LocalAI integration (via LocalAI)](https://apps.nextcloud.com/apps/integration_openai) | Yellow | Yes | Yes - e.g. Llama models by Meta | No | Yes | | | [OpenAI and LocalAI integration (via Ollama)](https://apps.nextcloud.com/apps/integration_openai) | Yellow | Yes | Yes - e.g. Llama models by Meta | No | Yes | | | [OpenAI and LocalAI integration (via IONOS AI Model Hub)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via Plusserver)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via Groqcloud)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via MistralAI)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [IBM watsonx.ai integration (via IBM watsonx.ai as a Service)](https://apps.nextcloud.com/apps/integration_watsonx) | Orange | No | Yes - e.g. Granite models by IBM | No | No | | | [IBM watsonx.ai integration (via IBM watsonx.ai software)](https://apps.nextcloud.com/apps/integration_watsonx) | Orange | No | Yes - e.g. Granite models by IBM | No | Yes | | Machine translation | [Local Machine Translation 2 (ExApp)](https://apps.nextcloud.com/apps/translate2) | Green | Yes | Yes - MADLAD models by Google | Yes | Yes | | | [DeepL integration](https://apps.nextcloud.com/apps/integration_deepl) | Red | No | No | No | No | | | [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) | Red | No | No | No | No | | | [OpenAI and LocalAI integration (via LocalAI)](https://apps.nextcloud.com/apps/integration_openai) | Green | Yes | Yes | Yes | Yes | | | [OpenAI and LocalAI integration (via Ollama)](https://apps.nextcloud.com/apps/integration_openai) | Yellow | Yes | Yes - e.g. Llama models by Meta | No | Yes | | | [OpenAI and LocalAI integration (via IONOS AI Model Hub)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via Plusserver)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via Groqcloud)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via MistralAI)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | Speech to Text | [Local Whisper Speech-To-Text 2 (ExApp)](https://apps.nextcloud.com/apps/stt_whisper2) | Yellow | Yes | Yes - Whisper models by OpenAI | No | Yes | | | [OpenAI and LocalAI integration](https://apps.nextcloud.com/apps/integration_openai) | Yellow | Yes | Yes - Whisper models by OpenAI | No | No | | | [OpenAI and LocalAI integration (via LocalAI)](https://apps.nextcloud.com/apps/integration_openai) | Green | Yes | Yes | Yes | Yes | | | [OpenAI and LocalAI integration (via Ollama)](https://apps.nextcloud.com/apps/integration_openai) | Yellow | Yes | Yes - e.g. Whisper | No | Yes | | | [OpenAI and LocalAI integration (via IONOS AI Model Hub)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via Plusserver)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via Groqcloud)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via MistralAI)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [Replicate integration](https://apps.nextcloud.com/apps/integration_replicate) | Yellow | Yes | Yes - Whisper models by OpenAI | No | No | | Image generation | [Local Stable Diffusion 2 (ExApp)](https://apps.nextcloud.com/apps/text2image_stablediffusion2) | Yellow | Yes | Yes - StableDiffusion XL model by StabilityAI | No | Yes | | | [Replicate integration](https://apps.nextcloud.com/apps/integration_replicate) | Yellow | Yes | Yes - StableDiffusion models by StabilityAI | No | No | | | [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) | Red | No | No | No | No | | | [OpenAI and LocalAI integration (via LocalAI)](https://apps.nextcloud.com/apps/integration_openai) | Green | Yes | Yes | Yes | Yes | | | [OpenAI and LocalAI integration (via Ollama)](https://apps.nextcloud.com/apps/integration_openai) | Yellow | Yes | Yes - e.g. Llama models by Meta | No | Yes | | | [OpenAI and LocalAI integration (via IONOS AI Model Hub)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via Plusserver)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via Groqcloud)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration (via MistralAI)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | Context Chat | [Nextcloud Assistant Context Chat](https://apps.nextcloud.com/apps/context_chat) | Yellow | Yes | Yes | No | Yes | | | [Nextcloud Assistant Context Chat (Backend)](https://apps.nextcloud.com/apps/context_chat_backend) | Yellow | Yes | Yes | No | Yes | | Context Chat Search | [Nextcloud Assistant Context Chat](https://apps.nextcloud.com/apps/context_chat) | Yellow | Yes | Yes | No | Yes | | Context Agent | [Nextcloud Context Agent (ExApp)](https://apps.nextcloud.com/apps/context_agent) | Green | Yes | Yes | Yes | Yes | | Text To Speech | [Open AI Text To Speech](https://apps.nextcloud.com/apps/integration_openai) | Red | No | No | No | No | | | [Local Text To Speech (ExApp)](https://apps.nextcloud.com/apps/text2speech_kokoro) | Yellow | Yes | Yes | No | Yes | | Document generation | [Nextcloud Office](https://apps.nextcloud.com/apps/richdocuments) | Green | Yes | Yes | Yes | Yes | | Live Transcription | [Local Live Transcription](https://apps.nextcloud.com/apps/live_transcription) | Yellow | Yes | Yes | No | Yes | | Optical Character Recognition | [Local OCR: PaddleOCR](https://apps.nextcloud.com/apps/ocr_paddle) | Yellow | Yes | Yes | No | Yes | | | [OpenAI and LocalAI integration \>= v5.0.0 (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) | Red | No | No | No | No | | | [OpenAI and LocalAI integration \>= v5.0.0 (via LocalAI)](https://apps.nextcloud.com/apps/integration_openai) | Green | Yes | Yes | Yes | Yes | | | [OpenAI and LocalAI integration \>= v5.0.0 (via Ollama)](https://apps.nextcloud.com/apps/integration_openai) | Yellow | Yes | Yes - e.g. Llama models by Meta | No | Yes | | | [OpenAI and LocalAI integration \>= v5.0.0 (via IONOS AI Model Hub)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration \>= v5.0.0 (via Plusserver)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration \>= v5.0.0 (via Groqcloud)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | | | [OpenAI and LocalAI integration \>= v5.0.0 (via MistralAI)](https://apps.nextcloud.com/apps/integration_openai) | Orange | No | Yes | No | No | ## Ethical AI Rating Until Hub 3, we succeeded in offering features without relying on proprietary blobs or third party services. Yet, while there is a large community developing ethical, safe and privacy-respecting technologies, there are many other relevant technologies users might want to use. We want to provide users with these cutting-edge technologies – but also be transparent. For some use cases, ChatGPT might be a reasonable solution, while for more private, professional or sensitive data, it is paramount to have a local, on-prem, open solution. To differentiate these, we developed an Ethical AI Rating. The rating has four levels: - Red - Orange - Yellow - Green It is based on points from these factors: - Is the software (both for inferencing and training) under a free and open source license? - Is the trained model freely available for self-hosting? - Is the training data available and free to use? If all of these points are met, we give a Green label. If none are met, it is Red. If 1 condition is met, it is Orange and if 2 conditions are met, Yellow. ## Features used by other apps Some of our AI features are realized as generic APIs that any app can use and any app can provide an implementation for by registering a provider. So far, these are Machine translation, Speech-To-Text, Text-To-Speech, Image generation, Text processing and Context Chat. ### Text processing As you can see in the table above we have multiple apps offering text processing using Large language models. In downstream apps like Context Chat and assistant, users can use the text processing functionality regardless of which app implements it behind the scenes. #### Frontend apps - [Assistant](https://apps.nextcloud.com/apps/assistant) for offering a graphical UI for the various tasks, a smart picker and "Chat with AI" functionality - [Mail](https://apps.nextcloud.com/apps/mail) for summarizing mail threads (see [the Nextcloud Mail docs](/link/1038) for how to enable this) - [Summary Bot](https://apps.nextcloud.com/apps/summary_bot) for summarizing chat histories in [Talk](https://apps.nextcloud.com/apps/spreed) - [Talk](https://apps.nextcloud.com/apps/spreed) for summarizing chat history (see [Nextcloud Talk docs](https://nextcloud-talk.readthedocs.io/en/latest/settings/#app-configuration) for how to enable this) - [Text](https://apps.nextcloud.com/apps/text) for offering an inline graphical UI for the various tasks - [Collectives](https://apps.nextcloud.com/apps/collectives) integrating Assistant through the smart picker to offer a graphical UI for the various tasks - [Notes](https://apps.nextcloud.com/apps/notes) integrating Assistant through the smart picker to offer a graphical UI for the various tasks - [Whiteboard](https://apps.nextcloud.com/apps/whiteboard) integrating Assistant through the smart picker to offer a graphical UI for the various tasks - [Deck](https://apps.nextcloud.com/apps/deck) integrating Assistant through the smart picker to offer a graphical UI for the various tasks - [Nextcloud Office](https://apps.nextcloud.com/apps/richdocuments) integrating Assistant through the smart picker to offer a graphical UI for the various tasks in documents - [Desktop Clients](/link/865) for simple "Chat with AI" #### Backend apps - [llm2](/link/995) - Runs open source AI LLM models on your own server hardware (Customer support available upon request) - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) - [IBM watsonx.ai integration (via IBM watsonx.ai as a Service)](https://apps.nextcloud.com/apps/integration_watsonx) - Integrates with the IBM watsonx.ai API to provide AI functionality from IBM Cloud servers (Customer support available upon request; see [AI as a Service](/link/1005)) ### Machine translation As you can see in the table above we have multiple apps offering machine translation capabilities. Each app brings its own set of supported languages. In downstream apps like the Text app, users can use the translation functionality regardless of which app implements it behind the scenes. #### Frontend apps - [Assistant](https://apps.nextcloud.com/apps/assistant) offering a graphical translation UI - [Analytics](https://apps.nextcloud.com/apps/analytics) for translating graph labels - [Talk](https://apps.nextcloud.com/apps/spreed) for translating messages and live translations in calls in conjunction with the [Live Transcription app](/link/1003) - [Deck](https://apps.nextcloud.com/apps/deck) for offering a translation UI in the card description - *Text* for offering the translation menu - [Notes](https://apps.nextcloud.com/apps/notes) for offering a translation UI in the note content - [Collectives](https://apps.nextcloud.com/apps/collectives) for offering a translation UI in the page content - [Whiteboard](https://apps.nextcloud.com/apps/whiteboard) for offering a translation UI through the assistant - [Nextcloud Office](https://apps.nextcloud.com/apps/richdocuments) for offering translation UI in the document content #### Backend apps - [translate2 (ExApp)](/link/994) - Runs open source AI translation models locally on your own server hardware (Customer support available upon request) - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) - [DeepL integration](http://apps.nextcloud.com/apps/integration_deepl) - Integrates with the deepl API to provide translation functionality from Deepl.com servers (Only community supported) ### Speech-To-Text As you can see in the table above we have multiple apps offering Speech-To-Text capabilities. In downstream apps like the Talk app, users can use the transcription functionality regardless of which app implements it behind the scenes. #### Frontend apps - [Assistant](https://apps.nextcloud.com/apps/assistant) offering a graphical translation UI, a smart picker and Audio Chat - [Talk](https://apps.nextcloud.com/apps/spreed) for transcribing calls (see [Nextcloud Talk docs](https://nextcloud-talk.readthedocs.io/en/latest/settings/#app-configuration) for how to enable this) #### Backend apps - [stt_whisper2](/link/996) - Runs open weights AI Speech-To-Text models on your own server hardware (Customer support available upon request) - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) ### Image generation As you can see in the table above we have multiple apps offering Image generation capabilities. In downstream apps like the Text-to-Image helper app, users can use the image generation functionality regardless of which app implements it behind the scenes. #### Frontend apps - [Assistant](https://apps.nextcloud.com/apps/assistant) for offering a graphical UI and a smart picker - [Deck](https://apps.nextcloud.com/apps/deck) for inserting images with the smart picker - Text for inserting images with the assistant and smart picker - [Notes](https://apps.nextcloud.com/apps/notes) for inserting images with the assistant and smart picker - [Collectives](https://apps.nextcloud.com/apps/collectives) for inserting images with the assistant and smart picker - [Whiteboard](https://apps.nextcloud.com/apps/whiteboard) for inserting images with the assistant and smart picker, generating diagrams and flowcharts with Mermaid - [Nextcloud Office](https://apps.nextcloud.com/apps/richdocuments) for inserting images with the assistant and smart picker #### Backend apps - [Local Stable Diffusion 2 (ExApp)](https://apps.nextcloud.com/apps/text2image_stablediffusion2) (Customer support available upon request) - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) - *integration_replicate* - Integrates with the replicate API to provide AI functionality from replicate servers (see [AI as a Service](/link/1005)) ### Text-To-Speech As you can see in the table above we have multiple apps offering speech generation capabilities. In downstream apps like the assistant app, users can use the speech generation functionality regardless of which app implements it behind the scenes. #### Frontend apps - [Assistant](https://apps.nextcloud.com/apps/assistant) for offering a audio chat #### Backend apps - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) - [Local Text To Speech (ExApp)](https://apps.nextcloud.com/apps/text2speech_kokoro) (Customer support available upon request) ### Context Chat Our Context Chat feature was introduced in Nextcloud Hub 7 (v28). It allows asking questions to the assistant related to your documents in Nextcloud. You will need to install both the context_chat app as well as the context_chat_backend External App. Be prepared that things might break or be a little rough around the edges. We look forward to your feedback! #### Frontend apps - [Assistant](https://apps.nextcloud.com/apps/assistant) for offering a graphical UI for the context chat tasks - [Nextcloud Context Agent](https://apps.nextcloud.com/apps/context_agent) for offering Context chat as a tool that the agent can execute in the "Chat with AI" feature #### Backend apps - [context_chat + context_chat_backend](/link/999) - (Customer support available upon request) #### Provider apps Apps can integrate their content with Context Chat to make it available for querying using Context Chat. The following apps have implemented this integration so far: - *files* - [Analytics](https://apps.nextcloud.com/apps/analytics) - [Mail](https://apps.nextcloud.com/apps/mail) (coming soon) - [Bookmarks](https://apps.nextcloud.com/apps/bookmarks) ### Context Chat Search Our Context Chat Search feature allows searching through your documents using natural language. You will need to install both the context_chat app as well as the context_chat_backend External App. We look forward to your feedback! #### Frontend apps - [Assistant](https://apps.nextcloud.com/apps/assistant) for offering a graphical UI for the context chat search tasks #### Backend apps - [context_chat + context_chat_backend](/link/999) - (Customer support available upon request) #### Provider apps See *Context Chat* section above. ### Context Agent Our Context Agent feature was introduced in Nextcloud Hub 9 (v30). It allows asking the assistant to execute tasks related to Nextcloud. You will need to install both the context_agent app as well as a text processing provider. #### Frontend apps - [Assistant](https://apps.nextcloud.com/apps/assistant) for offering a graphical UI for the "Chat with AI" feature #### Backend apps - [Nextcloud Context Agent](https://apps.nextcloud.com/apps/context_agent) for agentic AI capabilities in the "Chat with AI" feature (Customer support available upon request) #### Provider apps - [llm2](/link/995) - Runs open source AI LLM models on your own server hardware (Customer support available upon request) - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) ### Document generation Since Hub 11 you can let Nextcloud automatically generate Office documents with content. This functionality is available in the assistant app and made possible by the Nextcloud Office app. #### Frontend apps - [Assistant](https://apps.nextcloud.com/apps/assistant) for offering a graphical UI for the context chat search tasks #### Backend apps - [Nextcloud Office](https://apps.nextcloud.com/apps/richdocuments) #### Provider apps - [llm2](/link/995) - Runs open source AI LLM models on your own server hardware (Customer support available upon request) - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) - [IBM watsonx.ai integration (via IBM watsonx.ai as a Service)](https://apps.nextcloud.com/apps/integration_watsonx) - Integrates with the IBM watsonx.ai API to provide AI functionality from IBM Cloud servers (Customer support available upon request; see [AI as a Service](/link/1005)) ### Live transcription Since Hub 25 Autumn you can let Nextcloud automatically generate subtitles for video and audio calls in Nextcloud Talk. #### Frontend apps - [Talk](https://apps.nextcloud.com/apps/spreed) for displaying the subtitles in calls #### Backend apps - [live_transcription](/link/1003) - Runs open weights AI Speech-To-Text models on your own server hardware (Customer support available upon request) ### Optical Character Recognition As you can see in the table above we have multiple apps offering OCR capabilities. In downstream apps like the assistant app, users can use OCR functionality regardless of which app implements it behind the scenes. #### Frontend apps - [Assistant](https://apps.nextcloud.com/apps/assistant) #### Backend apps - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) \>= v5.0.0 - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) - [Local OCR: PaddleOCR (ExApp)](/link/1004) - (Customer support available upon request) ### Windmill workflows You can use the AI endpoints in your [Windmill workflows](/link/1032). Some example workflows showcasing the possibilities of combining Windmill workflows with Nextcloud AI are available on the [Windmill Hub](https://hub.windmill.dev/integrations/nextcloud/flows) ## Improve AI task pickup speed Most AI tasks will be run as part of the background job system in Nextcloud which only runs jobs every 5 minutes by default. To pick up scheduled jobs faster you can set up background job workers inside your Nextcloud main server/container that process AI tasks as soon as they are scheduled. If the PHP code or the Nextcloud settings values are changed while a worker is running, those changes won't be effective inside the runner. For that reason, the worker needs to be restarted regularly. It is done with a timeout of N seconds which means any changes to the settings or the code will be picked up after N seconds (worst case scenario). This timeout does not, in any way, affect the processing or the timeout of the AI tasks. > *Changed in version 32.0.7:* The command to run the worker has changed from `background-job:worker` to `taskprocessing:worker`. If you are running an older version of Nextcloud, please use the old command. ### Screen or tmux session Run the following occ command inside a screen or a tmux session, preferably 4 or more times for parallel processing of multiple requests by different or the same user (and as a requirement for some apps like context_chat). It would be best to run one command per screen session or per tmux window/pane to keep the logs visible and the worker easily restartable. set -e; while true; do sudo -E -u www-data php occ taskprocessing:worker -v -t 60; done For Nextcloud-AIO you should use this command on the host server. set -e; while true; do docker exec -it nextcloud-aio-nextcloud sudo -E -u www-data php occ taskprocessing:worker -v -t 60; done You may want to adjust the number of workers and the timeout (in seconds) to your needs. The logs of the worker can be checked by attaching to the screen or tmux session. ### Systemd service 1. Create a systemd service file in `/etc/systemd/system/nextcloud-ai-worker@.service` with the following content: [Unit] Description=Nextcloud AI worker %i After=network.target [Service] ExecStart=/opt/nextcloud-ai-worker/taskprocessing.sh %i Restart=always StartLimitInterval=60 StartLimitBurst=10 [Install] WantedBy=multi-user.target 2. Create a shell script in `/opt/nextcloud-ai-worker/taskprocessing.sh` with the following content and make sure to make it executable: #!/bin/sh echo "Starting Nextcloud AI Worker $1" cd /path/to/nextcloud sudo -E -u www-data php occ taskprocessing:worker -v -t 60 You may want to adjust the timeout to your needs (in seconds). 3. Enable and start the service 4 or more times: for i in {1..4}; do systemctl enable --now nextcloud-ai-worker@$i.service; done The status of the workers can be checked with (replace 1 with the worker number): systemctl status nextcloud-ai-worker@1.service The list of workers can be checked with: systemctl list-units --type=service | grep nextcloud-ai-worker The complete logs of the workers can be checked with (replace 1 with the worker number): journalctl -xeu nextcloud-ai-worker@1.service -f ## Frequently Asked Questions ### Why is my prompt slow? Reasons for slow performance from a user perspective can be > - Using CPU processing instead of GPU (sometimes this limit is imposed by the used app) > - High user demand for the feature: User prompts and AI tasks are usually processed in the order they are received, which can cause delays when a lot of users access these features at the same time. # Nextcloud Assistant Nextcloud assistant is the primary graphical user interface for interacting with artificial intelligence features in Nextcloud. It offers the graphical user interface for the unified AI Task processing API offering features like summarizing text, generating headlines, asking arbitrary questions, transcription of media files, image generation and it integrates with the context_chat app to offer in-context answers about your own data stored in Nextcloud. The assistant app also offers a chat interface to interact with the chosen language model, including multimodal chat with file attachments when a compatible backend is configured. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities. Find the user documentation here: [AI assistant user documentation](/link/793) ## Installation You can install the *assistant* app via the "Apps" page in Nextcloud, or by executing sudo -E -u www-data php occ app:enable assistant ## App store You can also find the app in our app store, where you can write a review: ## Repository You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: Nextcloud customers should file bugs directly with our Customer Support. ## Related apps Artificial intelligence at Nextcloud is built in a modular way, allowing you to choose from a variety of solutions for your needs. In order to make use of the various features of the Assistant you will need additional apps that act as backends to provide the actual implementation of the AI functionality. In the Nextcloud administration settings under "Artificial intelligence" you can select which AI backend app to use for which tasks. Note that some of the backend apps are only community maintained, while others are available for Customer support upon request. The AI admin settings will show all types of Assistant Tasks that are implemented by all your installed apps. Task types can be disabled in the AI admin settings so they are not available for the Assistant or other apps even if they are implemented. All implemented Task types are enabled by default. **Note**: At Nextcloud we focus on creating on-premise AI apps that run fully self-hosted on your own servers in order to preserve your privacy and data sovereignty. However, you can also offload these resource-heavy tasks to an ["AI as a Service" provider](/link/1005). **Note**: When using our on-premise AI apps, make sure you have a GPU with enough VRAM that fits all the features you need. For each app documented here you will find its hardware requirements. ### Machine translation In order to make use of machine translation features in the assistant, you will need an app that provides a translation backend: \* [translate2 (ExApp)](/link/994) - Runs open source AI translation models locally on your own server hardware (Customer support available upon request) \* *integration_deepl* - Integrates with the deepl API to provide translation functionality from Deepl.com servers (Only community supported) ### Speech-To-Text In order to make use of Speech-to-Text, you will need an app that provides a Speech-To-Text backend: - [stt_whisper2](/link/996) - Runs open source AI Speech-To-Text models on your own server hardware (Customer support available upon request) - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) ### Text processing In order to make use of text processing features in the assistant, you will need an app that provides a Text processing backend: - [llm2](/link/995) - Runs open source AI language models locally on your own server hardware (Customer support available upon request) - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) - *integration_watsonx* - Integrates with the IBM watsonx.ai API to provide AI functionality from IBM Cloud servers (Customer support available upon request; see [AI as a Service](/link/1005)) These apps currently implement the following Assistant Tasks: - *Generate text* (Tested with OpenAI GPT-3.5 and Llama 3.1 8B) - *Summarize* (Tested with OpenAI GPT-3.5 and Llama 3.1 8B) - *Generate headline* (Tested with OpenAI GPT-3.5 and Llama 3.1 8B) - *Extract topics* (Tested with OpenAI GPT-3.5 and Llama 3.1 8B) - *Context write* (Tested with OpenAI GPT-3.5 and Llama 3.1 8B) - *Reformulate text* (Tested with OpenAI GPT-3.5 and Llama 3.1 8B) These tasks may work with other models, but we can give no guarantees. ### Text-To-Image In order to make use of Text-To-Image features, you will need an app that provides an image generation backend: - [tex2image_stablediffusion2](/link/997) (Customer support available upon request) - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) - *integration_replicate* - Integrates with the replicate API to provide AI functionality from replicate servers (see [AI as a Service](/link/1005)) ### Context Chat In order to make use of our special Context Chat feature, offering insights about your own documents and data stored in Nextcloud, you will need the following apps: - [context_chat + context_chat_backend](/link/999) - (Customer support available upon request) You will also need a text processing provider as specified above (ie. llm2, integration_openai or integration_watsonx). ### Chat In order to make use of our "Chat with AI" feature you will need any one of the following apps: - [llm2](/link/995) - Runs open source AI language models locally on your own server hardware (Customer support available upon request) - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) ### Multimodal chat Multimodal chat extends "Chat with AI" so users can attach files for the language model to analyze in the conversation, such as images and, with some models, audio and video. Text files can also be attached as context. In order to make use of multimodal chat you will need any one of the following backends with a multimodal-capable model: - [llm2](/link/995) - Use a multimodal model such as the shipped Qwen 3.5 9B (vision) or Gemma 4 E4B (vision and audio) (Customer support available upon request) - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with multimodal models via the OpenAI API (Customer support available upon request; see [AI as a Service](/link/1005)) ### Voice Chat In order to make use of our "Audio chat" feature which allows you to interact with the assistant Chat via your Voice and Ears as if in a real conversation, you will need any of the following set of apps: - Either - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) - Or - [llm2](/link/995) - Runs open source AI language models locally on your own server hardware (Customer support available upon request) - [stt_whisper2](/link/996) - Runs open source AI Speech-To-Text models on your own server hardware (Customer support available upon request) - [text2speech_kokoro](/link/1002) - Runs open source AI Text-To-Speech models on your own server hardware (Customer support available upon request) ### Context Agent In order to make use of our AI agent feature, offering the execution of actions on behalf of the user based on the AI chat, you will need the following apps: - [context_agent](/link/1000) - (Customer support available upon request) You will also need a text processing provider as specified above (ie. *llm2* or *integration_openai*). ### Text-To-Speech In order to make use of Text-To-Speech, you will need an app that provides a Text-To-Speech backend, which is one of the following: - [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see [AI as a Service](/link/1005)) - [text2speech_kokoro](/link/1002) - Runs open source AI Text-To-Speech models on your own server hardware (Customer support available upon request) ## Configuration The Assistant admin settings can be found under the "Artificial intelligence" section. You can disable the assistant top menu entry there. You can also disable the AI-related smart pickers. The occ commands to change the options are listed below. ### Assistant configuration 1. Top-right Assistant occ config:app:set assistant assistant_enabled --value=1 --type=string To enable/disable the assistant button in the navigation bar for all users. 2. AI text generation smart picker occ config:app:set assistant free_prompt_picker_enabled --value=1 --type=string To enable/disable the AI text generation smart picker for all the users. 3. Text-to-image smart picker occ config:app:set assistant text_to_image_picker_enabled --value=1 --type=string To enable/disable the text-to-image smart picker for all the users. 4. Speech-to-text smart picker occ config:app:set assistant speech_to_text_picker_enabled --value=1 --type=string To enable/disable the speech-to-text smart picker for all the users. 5. Data folder occ config:app:set assistant default_data_folder --value="Assistant" --type=string The assistant stores the content it generates, such as generated images and speech-to-text output, in a folder in the files of each user. This sets the name that folder is given when it is created. It defaults to `Assistant`, and is also editable in the Assistant admin settings. Users can choose their own name in their personal Assistant settings, which is stored per user and takes precedence over the server-wide default: occ user:setting $USER_ID assistant data_folder "Assistant" Two things to keep in mind: - Changing either value does not rename or move a folder that already exists. It only affects the name used the next time a folder has to be created, so existing generated content stays where it is. - A user who already has a folder under the built-in name `Assistant`, from before a default was set, keeps using it. This avoids starting a second folder for them and leaving their earlier output behind. ### Task processing 1. List Tasks occ taskprocessing:task:list lists all task processing tasks. 2. Get Task occ taskprocessing:task:get $TASK_ID shows all information for a specific task. 3. Enable or disable a Task type occ taskprocessing:task-type:set-enabled $TASK_TYPE_ID 1 Set 1 to enable and 0 to disable an implemented task type. 4. Get Task statistics occ taskprocessing:task:stats shows statistics for all task processing Tasks. 5. Clean-up old tasks occ taskprocessing:task:cleanup delete tasks that are older than this number of seconds, defaults to 4 months. ### Image storage Days until generated images are deleted if they are not viewed. occ config:app:set assistant max_image_generation_idle_time --value=90 --type=integer ### Chat with AI 1. Chat User Instructions for Chat Completions occ config:app:set assistant chat_user_instructions --value="hello world" The user instructions that are prepended before the chat messages for the AI model to understand the context of the block of text. This is a good place not only to instruct the AI model to be polite and kind but also to for example answer all the queries in a particular language or better yet, follow the user's language. The sky is the limit. **Note**: The default instructions are optimized to work well across a variety of language models, but may not be optimal for the specific model you choose. Specifically, the model may be tempted to mention the user's name a bit too often and may mention the user's language in an unusual manner. 2. Chat User Instructions for Title Generation occ config:app:set assistant chat_user_instructions_title --value="hello title" This field is appended to the block of chat messages, i.e. attached after the messages. It is done this way to allow it to be used even with text completion models which could have the instructions as "The title for the above conversation could be "". 3. Last N messages to consider for chat completions occ config:app:set assistant chat_last_n_messages --value=10 The number of latest messages to consider for generating the next message. This does not include the user instructions, which is always considered in addition to this. This value should be adjusted in case you are hitting the token limit in your conversations too often. The AI text generation provider should ideally handle the max token limit case. ### Improve AI task pickup speed See [the relevant section in AI Overview](/link/992) for more information. # App: Local Machine translation 2 (translate2) The *translate2* app is one of the apps that provide machine translation functionality in Nextcloud and act as a translation backend for the [Nextcloud Assistant app](/link/993). The *translate2* app specifically runs only open source models and does so entirely on-premises. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities. The app currently supports 400+ languages. See the complete list here: ## Requirements - Minimal Nextcloud version: 30 - This app is built as an External App and thus depends on AppAPI v3.1.0 or higher - Nextcloud AIO is supported - We currently support NVIDIA GPUs and x86_64 CPUs - CUDA \>= v12.2.2 on your host system - GPU Sizing > - A NVIDIA GPU with at least 4 GB VRAM > - At least 6 GB of system RAM - CPU Sizing > - x86 CPU with 4-8 cores for the app to use (The more cores the faster it will be) > - At least 6 GB of RAM for the app should be enough (includes software+libraries and the model) ### Space usage > - ~ 2.95 GB for the docker container > - ~ 2.77 GB for the default model ## Installation 0. Make sure the [Nextcloud Assistant app](/link/993) is installed 1. [Install AppAPI and setup a Deploy Demon](/link/985) 2. Install the "Local Machine Translation" (translate2) ExApp via the "Apps" page in the Nextcloud web admin user interface ## Model Switch 1. Remove `hf_model_path` key from `loader` object in the `config.json` file in the docker container named `nc_app_translate2`. 2. Change `model_name` to the new model name to `Nextcloud-AI/madlad400-7b-mt-bt-ct2-int8_float32`. 3. Restart the docker container `docker restart nc_app_translate2` ## App store You can also find the app in our app store, where you can write a review: ## Repository You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: Nextcloud customers should file bugs directly with our Customer Support. ## Ethical AI Rating ### Rating: 🟢 Positive: \* the software for training and inference of this model is open source \* the trained model is freely available, and thus can be run on-premises \* the training data is freely available, making it possible to check or correct for bias or optimise the performance and CO2 usage. Learn more about the Nextcloud Ethical AI Rating [in our blog](https://nextcloud.com/blog/nextcloud-ethical-ai-rating). ## Known Limitations - AI translations are not a replacement for human professional translations and in many cases post-editing is required. AI translations can be used for understanding the main content of a text but not for translations that require special knowledge (such as technical content or legal content), or translations that require specific writing style to convey style, deeper meaning, or emotions (such as marketing content or translating books). - While the quality of the output will be fine for the most common languages (English, French, Spanish) the quality will suffer for languages that have less coverage in the original training set. - Make sure to test the translation model you are using it for whether it meets the use-case's quality requirements. The default model is the smallest of the batch and might produce duplicate translation outputs. Switch to a larger model if you need better quality and less artifacts, see [Model Switch](#bkmrk-model-switch). - Language models notoriously have a high energy consumption. - Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying models. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI). # App: Local large language model (llm2) The *llm2* app is one of the apps that provide text processing functionality using Large language models in Nextcloud and act as a text processing backend for the [Nextcloud Assistant app](/link/993), the *mail* app and [other apps making use of the core Text Processing API](/link/992). The *llm2* app specifically runs only open source models and does so entirely on-premises. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities. This app uses [llama.cpp](https://github.com/abetlen/llama-cpp-python) under the hood and is thus compatible with any model in *gguf* format. However, we primarily test with the models listed below. Output quality will differ depending on which model you use and downstream tasks like summarization or Context Chat may not work on other models. We thus recommend the following models: - [Qwen 3.5 9B](https://huggingface.co/unsloth/Qwen3.5-9B-GGUF) (good quality; multimodal vision; comes shipped with the app) - [Gemma 4 E4B](https://huggingface.co/google/gemma-4-E4B-it-qat-q4_0-gguf) (fast; multimodal vision and audio; comes shipped with the app) - [OLMo 3 7B Instruct](https://huggingface.co/allenai/Olmo-3-7B-Instruct) (fully open; green Ethical AI rating; text only; comes shipped with the app; see addendum below) ## Multimodal chat With a multimodal-capable model, *llm2* supports multimodal chat in the [Nextcloud Assistant app](/link/993): users can attach images (and with Gemma 4, also audio) to chat messages so the model can reason about their content. Text files can also be attached as context. The following shipped models support multimodality: - **Qwen 3.5 9B** - vision (images) - **Gemma 4 E4B** - vision (images) and audio Vision-capable models also register the *Analyze images* and *OCR* task types in addition to the usual text-processing tasks. Multimodal models need a matching multimodal projector (`mmproj`) *gguf* file next to the model file. The projector files for the shipped Qwen 3.5 and Gemma 4 models are downloaded automatically with the models. If you supply your own multimodal model, place its `mmproj` file in `/nc_app_llm2_data` as well and set `mmproj_path` in the model's JSON configuration (see [Configuring alternate models](/link/995) below). ## Multilinguality This app supports input and output in languages other than English if the underlying model supports the language. Qwen 3.5 supports the following 201 languages: \<\> Gemma 4 provides multilingual support, offering high-quality performance in 35 languages and basic support for 140 languages. Llama 3.1 [supports the following languages:](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct#multilingual-benchmarks) - English - Portuguese - Spanish - Italian - German - French - Hindi - Thai Note that other languages may work as well, but only the above languages are guaranteed to work with Llama 3.1. ## Requirements - This app is built as an External App and thus depends on AppAPI v3.1.0 or higher - Nextcloud AIO is supported - We currently support NVIDIA GPUs and x86_64 CPUs - CPU that supports AVX and AVX2 instruction - CUDA \>= v12.4 on your host system - GPU Sizing > - A NVIDIA GPU with at least 8GB VRAM > - At least 12GB of system RAM - CPU Sizing > - At least 12GB of system RAM > - The more cores you have and the more powerful the CPU the better, we recommend 10-20 cores > - The app will hog all cores by default, so it is usually better to run it on a separate machine ## Installation 0. Make sure the [Nextcloud Assistant app](/link/993) is installed 1. [Install AppAPI and setup a Deploy Demon](/link/985) 2. Install the "Local large language model" ExApp via the "Apps" page in the Nextcloud web admin user interface ### Supplying alternate models This app allows supplying alternate LLM models as *gguf* files in the `/nc_app_llm2_data` directory of the docker container. 1. Download a **gguf** model e.g. from huggingface 2. Copy the **gguf** file to `/nc_app_llm2_data` inside the docker container 3. Restart the llm2 ExApp 4. Select the new model in the Nextcloud AI admin settings ### Configuring alternate models Since every model requires slightly different inference parameters, you can pass along a configuration file for the alternate model files you supply. The configuration file for a model file must have the same name as the model file but must end in `.json` instead of `.gguf`. The strings `{system_prompt}` and `{user_prompt}` are variables that will be filled in by the app, so they must be part of your prompt template. Here is an example config file for Llama 2: ``` json { "prompt": "<|im_start|> system\n{system_prompt}\n<|im_end|>\n<|im_start|> user\n{user_prompt}\n<|im_end|>\n<|im_start|> assistant\n", "loader_config": { "n_ctx": 4096, "max_tokens": 2048, "stop": ["<|im_end|>"] } } ``` Here is an example configuration for Llama 3: ``` json { "prompt": "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>\n{user_prompt}<|eot_id|>\n<|start_header_id|>assistant<|end_header_id|>\n", "loader_config": { "n_ctx": 8000, "max_tokens": 4000, "stop": ["<|eot_id|>"], "temperature": 0.3 } } ``` ## Scaling It is currently not possible to scale this app, we are working on this. Based on our calculations an instance has a rough capacity of 1000 user requests per hour. However, this number is based on theory and we do appreciate real-world feedback on this. If you would like to scale up your language model usage, we recommend using an [AI as a Service provider](/link/1005) or hosting a service compatible with the OpenAI API yourself that can be scaled up and connecting nextcloud to it via the [integration_openai app](https://apps.nextcloud.com/apps/integration_openai). ## App store You can also find the app in our app store, where you can write a review: ## Repository You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: Nextcloud customers should file bugs directly with our Support system. ## Known Limitations - We currently only support languages that the underlying model supports; correctness of language use in languages other than English may be poor depending on the language's coverage in the model's training data (We recommend Qwen 3.5 for Asian languages, while Gemma 4 is better for European languages) - Language models can be bad at reasoning tasks - Language models can be bad at math - Language models are likely to generate false information and should thus only be used in situations that are not critical. It's recommended to only use AI at the beginning of a creation process and not at the end, so that outputs of AI serve as a draft for example and not as final product. Always check the output of language models before using it. - Make sure to test the language model you are using it for whether it meets the use-case's quality requirements - Language models notoriously have a high energy consumption, if you want to reduce load on your server you can choose smaller models or quantized models in exchange for lower accuracy - Multimodal features (chat with image or audio attachments, Analyze images, OCR) require a multimodal-capable model and its matching `mmproj` file; text-only models such as Llama 3.1 will not process image or audio attachments - Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI) ## Addendum: Running with a fully open model If you would like to use a fully open model that scores a green score on our Ethical AI rating, we recommend the following model: - Olmo 3 (either in 7B or 32B): ### What makes OLMo a fully open model? - The code for training, fine-tuning and inference of the model is publicly available and fully open source - The training data with which the model is pretrained is publicly available - The model itself is publicly available and fully open source - The instruction tuning data is publicly available - The Reinforcement learning model is publicly available and fully open source ### Limitations - OLMo currently only works well with English language input - In our tests it sometimes produced hallucinated or garbled output; make sure to thoroughly test the model for your use case - It cannot use tools, so cannot be used in conjunction with [Context Agent](/link/1000) # App: Local Whisper Speech-To-Text (stt_whisper2) The *stt_whisper2* app is one of the apps that provide Speech-To-Text functionality in Nextcloud and act as a media transcription backend for the [Nextcloud Assistant app](/link/993), the *talk* app and [other apps making use of the core Speech-To-Text API](/link/992). The *stt_whisper2* app specifically runs only open source models and does so entirely on-premises. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities. This app supports input and output in languages other than English if the underlying model supports the language. This app uses [faster-whisper](https://github.com/SYSTRAN/faster-whisper) under the hood. Output quality will differ depending on which model you use, we recommend the following models: > - OpenAI Whisper large v3 turbo (multilingual) > - OpenAI Whisper medium.en (English only) Whisper large v3 supports about ~100 languages and shows outstanding performance in ~10 of them. For more details see the [OpenAI Whisper paper](https://cdn.openai.com/papers/whisper.pdf) ## Requirements - Minimal Nextcloud version: 28 - This app is built as an External App and thus depends on AppAPI v2.3.0 - Nextcloud AIO is supported - We currently support NVIDIA GPUs and x86_64 CPUs - CUDA \>= v12.2 on your host system - GPU Sizing > - A NVIDIA GPU with at least 4GB VRAM - CPU Sizing > - The more cores you have and the more powerful the CPU the better, we recommend 10-20 cores > - The app will hog all cores by default, so it is usually better to run it on a separate machine > - 4GB for the app ## Installation 0. Make sure the [Nextcloud Assistant app](/link/993) is installed 1. [Install AppAPI and setup a Deploy Demon](/link/985) 2. Install the *stt_whisper2* "Local Speech-To-Text" ExApp via the "Apps" page in the Nextcloud web admin user interface ### Supplying alternate models This app allows supplying alternate models in the `/nc_app_stt_whisper2_data` directory of the docker container. You can use any [\*faster-whisper\* model by Systran on hugging face](https://huggingface.co/Systran) in the following way: 1. git cloning the respective repository 2. Copying the folder with the git repository to `/nc_app_stt_whisper2_data` inside the docker container. 3. Restarting the Whisper ExApp 4. Selecting the respective model in the Nextcloud AI admin settings ## Scaling It is currently not possible to scale this app, we are working on this. Based on our calculations an instance has a rough capacity of 4h of transcription throughput per minute (measured with 8 CPU threads on an Intel(R) Xeon(R) Gold 6226R). It is unclear how close to real-world usage this number is, so we do appreciate real-world feedback on this. ## App store You can also find this app in our app store, where you can write a review: ## Repository You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: Nextcloud customers should file bugs directly with our customer support. ## Known Limitations - We currently do not support live transcription - We currently only support languages supported by the underlying Whisper models - The whisper models perform unevenly across languages, and may show lower accuracy on low-resource and/or low-discoverability languages or languages where there was less training data available. The models also exhibit disparate performance on different accents and dialects of particular languages, which may include higher word error rate across speakers of different genders, races, ages, or other demographic criteria. - Language models are likely to generate false information and should thus only be used in situations that are not critical. It's recommended to only use AI at the beginning of a creation process and not at the end, so that outputs of AI serve as a draft for example and not as final product. Always check the output of language models before using it. - Make sure to test the language model you are using it for whether it meets the use-case's quality requirements - Language models notoriously have a high energy consumption, if you want to reduce load on your server you can choose smaller models or quantized models in exchange for lower accuracy - Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI) # App: Local Image Generation (text2image_stablediffusion2) The *text2image_stablediffusion2* app is one of the apps that provide image generation functionality in Nextcloud and act as an image generation backend for the [Nextcloud Assistant app](/link/993) and other apps making use of the image generation functionality. The *text2image_stablediffusion2* app specifically runs only open source models and does so entirely on-premises. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities. ## Requirements - This app is built as an External App and thus depends on AppAPI v3.1.0 or higher - Nextcloud AIO is supported - We currently support NVIDIA GPUs and x86_64 CPUs - CUDA \>= v12.2 on your host system - GPU Sizing > - A NVIDIA GPU with at least 8GB VRAM - CPU Sizing > - At least 8GB of system RAM > - The more cores you have and the more powerful the CPU the better, we recommend 10-20 cores > - The app will hog all cores by default, so it is usually better to run it on a separate machine ## Installation - Make sure the [Nextcloud Assistant app](/link/993) is installed - [Install AppAPI and setup a Deploy Demon](/link/985) - Install the "Local large language model" ExApp via the "Apps" page in the Nextcloud web admin user interface ## Scaling It is currently not possible to scale this app, we are working on this. Based on our calculations an instance has a rough capacity of 120 image requests per hour (each user request can be for multiple images). However, this number is based on theory and we do appreciate real-world feedback on this. ## App store You can also find the app in our app store, where you can write a review: ## Repository You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: Nextcloud customers should file bugs directly with our Support system. ## Known Limitations - The generated images are of a fixed resolution (512x512 pix), and the model does not achieve perfect photorealism - The model cannot render legible text - Faces and people in general may not be generated properly - The results for certain image generation requests can be biased and may enforce stereotypes - We currently only support languages that the underlying model supports; correctness of language use in languages other than English may be poor depending on the language's coverage in the model's training data - Make sure to test the app for whether it meets the use-case's quality requirements - Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI) # App: Recognize The *recognize* app provides media tagging and face recognition functionality for the memories app. *Recognize* can group similar faces on user's photos ("face recognition"); it can add fitting tags to photos detecting landscapes, food, vehicles, buildings animals and other objects, as well as known landmarks and monuments; it can recognize music genres in user's audio files and adds tags for those; it can recognize human actions on user's video files and add tags for them. It specifically runs only open source models and does so entirely on-premises. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities. The actual classification work can be carried out by one of two interchangeable backends (see [Classifier backends](#bkmrk-classifier-backends) below): the built-in Node.js/TensorFlow.js classifiers that run directly on your Nextcloud nodes, or the *recognize_backend* ExApp, which is deployed as a container via AppAPI and plugs into Nextcloud's TaskProcessing framework. ## Front-end Tagged files will appear in the Memories app under the "Tags" section as well as in the normal Files app. Face recognition results will appear under the "People" section in the Memories app. ## Classifier backends Regardless of the backend, *recognize* itself always does the same work: it crawls the file system, keeps a queue of files per model, schedules background jobs, writes the resulting tags and face detections to the database and clusters faces into persons. Only the classification step itself differs. ### Node.js backend (default) The classifiers ship with the app as TensorFlow.js models and are executed by spawning a Node.js child process on the Nextcloud node that runs the background job. The models have to be downloaded to each node with `occ recognize:download-models`, and Node.js (plus FFmpeg for video) must be available on those nodes. This is the traditional and default setup described in the rest of this page. ### recognize_backend ExApp (TaskProcessing) [recognize_backend](https://github.com/nextcloud/recognize_backend) is a separate Nextcloud ExApp that is deployed as a Docker container through [AppAPI](/link/985). It registers one TaskProcessing provider per classification task type, and *recognize* then hands off batches of files as TaskProcessing tasks instead of spawning local Node.js processes. Results are picked up asynchronously and applied to tags and face detections exactly as before. Compared to the Node.js backend this means: - No Node.js, no FFmpeg binary, no `occ recognize:download-models` and no TensorFlow.js on the Nextcloud nodes; the Nextcloud nodes only schedule and collect tasks - The classification load can be moved to a dedicated (GPU) machine, independent of the nodes that run cron - Newer, larger and generally more accurate models (see the table below), loaded through their canonical Python libraries (Hugging Face `transformers`, `insightface`) - GPU acceleration through CUDA, with automatic fall back to CPU if no usable GPU is present - Drawback: a sizeable container image and model downloads, and a hard dependency on AppAPI and a working deploy daemon The following task types and models are implemented by *recognize_backend*: | Task type | Recognize feature | Model | |----|----|----| | `recognize:image:classification` | Object recognition | ConvNeXt V2 (Large, 384, ImageNet-22k) | | `recognize:image:facerecognition` | Face recognition | InsightFace `buffalo_l` (SCRFD detector + ArcFace 512-dim embeddings) | | `recognize:audio:classification` | Music genre recognition | Audio Spectrogram Transformer (AudioSet, 527 classes), plus a dedicated music genre classifier for clips detected as music | | `recognize:video:classification` | Video action recognition | VideoMAE (Large, Kinetics-400) | Landmark recognition is currently **not** part of *recognize_backend*. Images that object recognition identifies as buildings are still queued for the Node.js landmarks classifier, so landmark recognition continues to require Node.js and `occ recognize:download-models` on the nodes running background jobs, even in TaskProcessing mode. If you do not want that, leave landmark recognition disabled. ## Requirements ### Common requirements - Background Jobs must be executed via cron ### Node.js backend - Nextcloud AIO is not supported but will likely work at sub optimal speed - Minimum supported Nextcloud version: 26 - x86 CPU - GNU lib C - Using GPU processing is supported, but not required; slow performance is expected if you are not using a GPU - We currently only support NVIDIA GPUs - For GPU support you need to install: > - NVIDIA® GPU drivers version 450.80.02 or higher. > - CUDA® Toolkit 11.x > - cuDNN SDK 8.x - GPU Sizing > - The models used by recognize require about 1GB of VRAM or less - CPU Sizing > - If you don't have a GPU, this app will utilize your CPU cores > - The more cores you have and the more powerful the CPU the better, we recommend 10-20 cores > - In the app settings you can set the number of cores to use > - At least ~4GB of RAM dedicated for recognize ### recognize_backend ExApp - Nextcloud v35 or later, and at least *recognize* v13 - The [AppAPI](https://apps.nextcloud.com/apps/app_api) app with a configured deploy daemon (Docker socket proxy or Harp proxy) - x86-64 host for the deploy daemon; the published container image is built for `linux/amd64` - Outbound HTTPS access from the container to `huggingface.co`, since the models are downloaded on first use - Using GPU processing is supported, but not required; expect slow performance on CPU, especially for video > - We currently only support NVIDIA GPUs > - NVIDIA® GPU drivers and the NVIDIA container toolkit must be installed on the deploy daemon host; the image ships CUDA 12.2 and cuDNN 8 > - GPU Sizing: about 4-6GB of VRAM if all four task types are enabled; models are loaded lazily on first use and then stay resident for the lifetime of the container > - If a CUDA/cuDNN runtime error occurs, the app permanently falls back to CPU for the remainder of the container's lifetime and logs a warning - CPU Sizing > - The more cores you have and the more powerful the CPU the better > - At least ~8GB of RAM dedicated to the container ### Disk space usage > - Node.js backend: ~1.5GB for all models in total, on every node that runs background jobs > - recognize_backend ExApp: ~9GB for the container image, plus ~3GB of model weights downloaded into the app's persistent storage volume on first use ## Installation ### Installation with the Node.js backend 1. Install the *recognize* app via the "Apps" page in Nextcloud, or by executing occ app:enable recognize 2. Execute the following command on your server terminal of each node that runs background jobs: occ recognize:download-models 3. Go to your Nextcloud Administration settings and open the *recognize* admin settings page 4. Enable all modes of operation that you want the app to undertake 5. Enable GPU mode if you have a GPU that you want to use; if you want to use CPU only, you can set the number of cores to use here 6. Execute the following command on your server terminal to stop background processing of existing files: occ recognize:clear-background-jobs 7. Execute the following command on your server terminal to process all existing files in bulk (This may take a long time, depending on how many files you have on your instance): occ recognize:classify 8. Execute the following command on your server terminal to calculate face clusters from faces found in all existing files (Run this repeatedly until no more clusters are found): occ recognize:cluster-faces 9. All new files from this point on will be automatically processed in background tasks without manual intervention ### Installation with the recognize_backend ExApp 1. Install and configure the *AppAPI* app and a deploy daemon as described in [AppAPI and External Apps](/link/985) 2. Install the *recognize* app via the "Apps" page in Nextcloud, or by executing occ app:enable recognize 3. Install the *Recognize Backend* ExApp from the "External Apps" page in your Nextcloud Administration settings. Note that the first deployment pulls a container image of roughly 9GB, so this can take a while. 4. Go to your Nextcloud Administration settings and open the *recognize* admin settings page. In the "Classifier backend" section, "Use Nextcloud TaskProcessing for classification" is switched on automatically as soon as the *recognize_backend* ExApp is installed and enabled; the Node.js, FFmpeg, WASM, GPU and resource usage sections disappear, because they no longer apply. 5. Enable all modes of operation that you want the app to undertake. If you want landmark recognition, you also have to run `occ recognize:download-models` on every node that runs background jobs, since landmarks are still classified locally. 6. Execute the following command on your server terminal to queue all existing files for classification (the actual work is then done by background jobs, which hand the files to the ExApp; this may take a long time, depending on how many files you have on your instance): occ recognize:recrawl Note that `occ recognize:classify`, which classifies files synchronously on the terminal, does **not** work in TaskProcessing mode. Bulk classification always goes through cron in this mode. 7. Execute the following command on your server terminal to calculate face clusters from faces found in all existing files (Run this repeatedly until no more clusters are found): occ recognize:cluster-faces 8. All new files from this point on will be automatically processed in background tasks without manual intervention The progress of running tasks is visible in the *recognize* admin settings, which shows the number of scheduled and running TaskProcessing tasks per model next to the queue counts. ### Switching between backends Object, audio and video tags produced by the two backends are compatible: they end up as ordinary system tags, and you can simply leave existing tags in place, or remove them with `occ recognize:reset-tags` and reclassify. Face detections are **not** compatible. The two backends use different face recognition models with different embeddings and different clustering distances, so detections created by one backend cannot be clustered together with detections created by the other. After switching the backend, reset the existing face data and let it be recomputed: > occ recognize:reset-faces > > occ recognize:recrawl > > occ recognize:cluster-faces (`occ recognize:reset-faces` removes all face detections and clusters, `occ recognize:recrawl` puts all files back into the queues, and clustering then runs over the newly created detections. With the Node.js backend you can use `occ recognize:classify` instead of `occ recognize:recrawl` to do the classification on the terminal rather than through cron.) ## Configuration of the recognize_backend ExApp The defaults are sensible for most instances. If you want to change models or thresholds, the following environment variables can be set on the ExApp, either in the deploy daemon UI or with `occ app_api:app:register` / your container runtime: - `RECOGNIZE_IMAGE_MODEL` — Hugging Face model id for object recognition (default: `facebook/convnextv2-large-22k-384`) - `RECOGNIZE_IMAGE_TOP_K` — maximum number of labels returned per image - `RECOGNIZE_IMAGE_THRESHOLD` — minimum probability for an image label to be returned - `RECOGNIZE_FACE_MODEL` — InsightFace model pack for face recognition (default: `buffalo_l`) - `RECOGNIZE_FACE_DET_SIZE` — square input size in pixels for the face detector (default: `640`); larger values find smaller faces at the cost of speed - `RECOGNIZE_AUDIO_MODEL` — Hugging Face model id for audio classification (default: `MIT/ast-finetuned-audioset-10-10-0.4593`) - `RECOGNIZE_AUDIO_TOP_K` — maximum number of categories returned per audio file (default: `5`) - `RECOGNIZE_AUDIO_THRESHOLD` — minimum probability for an audio category to be returned (default: `0.2`) - `RECOGNIZE_MUSIC_GENRE_ENABLED` — set to `0` to skip the dedicated music genre classifier even when the audio is detected as music (default: `1`) - `RECOGNIZE_MUSIC_GENRE_MODEL` — Hugging Face model id for music genre classification (default: `dima806/music_genres_classification`) - `RECOGNIZE_MUSIC_GENRE_TOP_K` — maximum number of genre labels appended for music clips (default: `3`) - `RECOGNIZE_MUSIC_GENRE_THRESHOLD` — minimum probability for a genre label to be appended (default: `0.25`) - `RECOGNIZE_MUSIC_DETECTION_THRESHOLD` — minimum probability on a "music"/"singing" label required to run the genre classifier at all (default: `0.3`) - `RECOGNIZE_VIDEO_MODEL` — Hugging Face model id for video classification (default: `MCG-NJU/videomae-large-finetuned-kinetics`) - `RECOGNIZE_VIDEO_TOP_K` — maximum number of categories returned per video (default: `5`) - `RECOGNIZE_VIDEO_THRESHOLD` — minimum probability for a video category to be returned (default: `0.15`) - `RECOGNIZE_VIDEO_FRAMES` — number of evenly spaced frames sampled per video clip (default: `16`) - `TASK_POLLING_INTERVAL` — seconds between polls for new TaskProcessing jobs when idle (default: `5`) Any Hugging Face model compatible with the corresponding `transformers` pipeline, and any InsightFace model pack, can be used. Note that the *recognize* app maps the returned labels onto its own tag vocabulary, so swapping in an unrelated model may produce labels that are dropped. ## Scaling With the Node.js backend it is possible to scale this app by adding multiple "background" nodes to your cluster that will only process background jobs by executing cron.php. With the *recognize_backend* ExApp, the Nextcloud nodes only schedule tasks and apply results, so the classification throughput is determined by the machine that hosts the ExApp container: give it a GPU, more CPU cores and more RAM to speed up processing. Files are handed over in batches of up to 500 per task, and the container processes one task at a time. You still need cron to run on your Nextcloud nodes so that files are crawled, queued and handed over. ## App store You can also find the app in our app store, where you can write a review: The ExApp backend is listed separately: ## Repository You can find the app's source repository on GitHub where you can report bugs and contribute fixes and features: The source of the ExApp backend lives in a separate repository: Nextcloud customers should file bugs directly with our Support system. ## Known Limitations - Make sure to test whether the functionality meets the use-case's quality requirements - Machine learning models notoriously have a high energy consumption - Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI) - When using the *recognize_backend* ExApp: > - Landmark recognition is not provided by the ExApp and still runs locally via Node.js > - Face detections created by the two backends are not interchangeable; switching the backend requires resetting and recomputing face data > - Models are downloaded from Hugging Face on first use, so the container needs internet access at least once, and the first task of each type is noticeably slower than subsequent ones ## Ethical AI Rating ### Node.js backend #### Rating for Photo object detection: Green Positive: - The software for training and inference of this model is open source - The trained model is freely available, and thus can be run on-premises - The training data is freely available, making it possible to check or correct for bias or optimize the performance and CO2 usage. #### Rating for Photo face recognition: Green Positive: - The software for training and inference of this model is open source - The trained model is freely available, and thus can be run on-premises - The training data is freely available, making it possible to check or correct for bias or optimize the performance and CO2 usage. #### Rating for Video action recognition: Green Positive: - The software for training and inferencing of this model is open source - The trained model is freely available, and thus can be ran on-premises - The training data is freely available, making it possible to check or correct for bias or optimize the performance and CO2 usage. #### Rating Music genre recognition: Yellow Positive: - The software for training and inference of this model is open source - The trained model is freely available, and thus can be run on-premises Negative: - The training data is not freely available, limiting the ability of external parties to check and correct for bias or optimise the model’s performance and CO2 usage. ### recognize_backend ExApp #### Rating: Yellow Positive: - The software for training and inference of all bundled models is open source - The trained models are freely available, and thus can be run on-premises Negative: - The training data of the models is not freely available, limiting the ability of external parties to check and correct for bias or optimise the models' performance and CO2 usage. Learn more about the Nextcloud Ethical AI Rating [in our blog](https://nextcloud.com/blog/nextcloud-ethical-ai-rating/). # App: Context Chat Context Chat is an [assistant](/link/993) feature that is implemented via an ensemble of two apps: > - the `context_chat` app, written purely in PHP > - the `context_chat_backend` ExternalApp written in Python Together they provide the ContextChat *text processing* and *search* tasks accessible via the [Nextcloud Assistant app](/link/993). The `context_chat` and `context_chat_backend` apps will use the configured text-to-text task processing provider, which is required on a fresh install. It can be configured to run open source models entirely on-premises, see the list of providers [here](/link/992) in the "Backend apps" section. This app supports input and output in the same languages that the currently configured text-to-text task processing provider supports. ## Requirements - Minimal Nextcloud version: 32 - Nextcloud AIO is supported - We currently support NVIDIA GPUs, AMD GPUs (via Vulkan) and x86_64 CPUs - CPU that supports AVX and AVX2 instruction - CUDA \>= v12.8 on your host system if NVIDIA GPUs are used - Both podman and docker are supported - GPU Setup Sizing > - A GPU with at least 2GB VRAM > - The requirements for the text-to-text providers should be checked separately for each app [here](/link/992) in the "Backend apps" section, as they can vary greatly based on the model used and whether the provider is hosted locally or remotely. > > - At least 8GB of system RAM > - 2 GB + additional 500MB for each concurrent request made to the backend if configuration parameters are changed - CPU Setup Sizing > - At least 12GB of system RAM > - 2 GB + additional 500MB for each additional concurrent query request > - 8 GB is recommended in the above case for the default settings > > - This app makes use of the configured text-to-text task processing provider instead of running its own language model by default, thus 4+ cores for the embedding model is needed - A dedicated machine is recommended ### Space usage This app employs a bundled DB with Vector support called [PostgreSQL](https://www.postgresql.org/). All the users' textual data is duplicated, chunked and stored on disk in this vector DB along with semantic embedding vectors for the content. ## Installation 1. Make sure the [Nextcloud Assistant app](/link/993) is installed 2. Setup a [Deploy Daemon](/link/985) in AppAPI Admin settings 3. Install the `context_chat_backend` ExApp via the "Apps" page in Nextcloud, or by executing (checkout the readme at for manual install steps) occ app_api:app:register context_chat_backend 4. Install the `context_chat` app via the "Apps" page in Nextcloud, or by executing occ app:enable context_chat 5. Install a text-to-text provider (text generation provider) via the "Apps" page in Nextcloud. A list of providers can be found [here](/link/992) in the "Backend apps" section. 6. Optionally but recommended, setup background workers for faster pickup of tasks. See [the relevant section in AI Overview](/link/992) for more information. > **提示** > > Both apps need to be installed and both major version and minor version of the two apps must match for the functionality to work (ie. "v1.3.4" and "v1.3.1"; but not "v1.3.4" and "v2.1.6"; and not "v1.3.4" and "v1.4.5"). Keep this in mind when updating. ## Initial loading of data ### Auto-indexing Context chat will automatically load user data into the Vector DB. Context chat backend pulls the queued files and content providers' items from the context chat PHP app and indexes them.\ The initial loading of data can take a long time depending on the number of files and their size. ## Scaling Listed below are the major parts of the system that can be scaled independently to improve performance: 1. The text-to-text task processing provider (from among the list of providers [here](/link/992) in the "Backend apps" section) The text-to-text task processing provider can be scaled by using a hosted service using the [OpenAI and LocalAI integration (via OpenAI API)](https://apps.nextcloud.com/apps/integration_openai) like OpenAI or by hosting your own model on powerful hardware. 2. The vector DB performance The vector DB performance can be scaled by using a dedicated or cluster setup for PostgreSQL with the pgvector extension.\ The connection string of the external vector DB can be set using the environment variable `EXTERNAL_DB` during deployment in the "Deploy Options". 3. The embedding model performance The embedding model performance can be scaled by using a hosted embedding service, locally or remotely hosted. It should be able to serve an OpenAI-compatible API.\ The embedding service URL can be set using the environment variable `CC_EM_BASE_URL` during deployment in the "Deploy Options". Other options like the model name, api key, or username and password can be set using the environment variables `CC_EM_MODEL_NAME`, `CC_EM_APIKEY`, `CC_EM_USERNAME`, and `CC_EM_PASSWORD` respectively. > **注意** > > The embedding model cannot be changed after installing the app. To use a different embedding model or service, you will need to do a full uninstall (removing all data of the ExApp) and reinstall the `context_chat_backend` ExApp with the new environment variables and an empty vector DB. If the vector DB is external, the connected database (database may be named `ccb`) should be dropped before installing the ExApp again. > > For the `context_chat` app, obtain a clean slate by dropping all the `_context_chat_*` tables in the database, and removing all the config values: And re-installing it. > > ``` sql > drop table if exists oc_context_chat_action_queue; > drop table if exists oc_context_chat_content_queue; > drop table if exists oc_context_chat_fs_events; > drop table if exists oc_context_chat_queue; > delete from oc_appconfig where appid = 'context_chat'; > ``` 4. The parsing of the documents to extract text The parsing of the documents to extract text is done in a single instance of the `context_chat_backend` ExApp in a docker-based environment. It is a CPU-bound task, so having a powerful CPU will help speed up the parsing process.\ This can be scaled by using Kubernetes for deployment, allowing multiple instances of the `context_chat_backend` ExApp to handle the parsing concurrently, see [the Kubernetes section](/link/999). If `context_chat_backend` is already deployed, you can change these environment variables by redeploying it with the new values. 1. Go to Apps page -\> search for "Context Chat Backend" 2. Disable and remove the app taking care the data is not removed (except when the embedding model is changed, in which case the data should be removed) 3. Set the "Deploy Options" with the new environment variables 4. Reinstall the app ### Kubernetes Starting with version 5.4.0 of the app and Nextcloud 34, Kubernetes is supported for deployment of the backend to scale ContextChat to large instances. Nextcloud customers can find details about Kubernetes deployment in the customer documentation or by contacting support. ## App store You can also find the `context_chat` app in our app store, where you can write a review: ## Repository You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: and Nextcloud customers should file bugs directly with our Customer Support. ## Commands (OCC) The options for each command can be found like this, using stats as example: `context_chat:stats --help` - `context_chat:prompt` Ask a question about your data, with options for selective context. - `context_chat:search` Perform a semantic (vector DB based) search on your indexed documents, with options for selective context. - `context_chat:stats` Shows the time taken to complete the initial indexing of the documents if it has finished,\ and the current no. of items in the indexer and actions queue.\ "Actions" refers to tasks like file deletions, ownership changes through share changes, etc.\ These file and ownership changes are synced with the backed through this actions queue. ## Configuration Options - `auto_indexing` string (default: 'true') To allow/disallow the IndexerJob from running in the background. Not required to be configured normally. occ config:app:set context_chat auto_indexing --value='true' --type=string Context chat backend's configuration options are available through the environment variable in [Deploy Options](/link/990). ## Logs Logs for both the `context_chat` PHP app and the `context_chat_backend` ExApp can be found in the admin settings of your Nextcloud GUI as well as in the Context Chat log file, which is usually located in the Nextcloud data directory. The log file is named `context_chat.log`. The internal PostgreSQL Vector DB logs (in case of non-Kubernetes setups) can be found inside the docker container at `/nc_app_context_chat_backend_data/vector_db_data/pgsql/logfile`. It can be copied to host using `docker cp nc_app_context_chat_backend:/nc_app_context_chat_backend_data/vector_db_data/pgsql/logfile /tmp/vectordb-logfile`.\ This might be needed when the automated setting up of the vector db fails with something like: "pg_ctl: could not start server". When running in Kubernetes, the logs for the backend will be in the respective pod's logs, which can be accessed using `kubectl logs ` command. ## Troubleshooting 1. If the docker container seems to suddenly restart during indexing or querying, it could be related to RAM/storage filling up, or AVX being unavailable on the system. AVX can be checked using `grep -i avx /proc/cpuinfo` command on the host system. If AVX is not available, the app will not work. 2. Look for issues in the diagnostic logs, the server logs and the docker container `nc_app_context_chat_container` logs. If unsure, open an issue in either of the repositories. 3. Check "Admin settings -\> Context Chat" for statistics and information about the indexing process. ## File access control rules not supported In Nextcloud you can set up file access control rules using the [files_accesscontrol](https://apps.nextcloud.com/apps/files_accesscontrol) app to restrict access to certain files. Context Chat does **not** follow these rules. It is thus possible for users who have been denied access to a document via the files_accesscontrol app to still gain access via Context Chat if the document is visible in the files app for the user in question. ## Known Limitations - Language models are likely to generate false information and should thus only be used in situations that are not critical. It's recommended to only use AI at the beginning of a creation process and not at the end, so that outputs of AI serve as a draft for example and not as final product. Always check the output of language models before using it and make sure whether it meets your use-case's quality requirements. - Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI). - Files larger than 100MB are not supported - Password protected PDFs or any other files are not supported. There will be error logs mentioning cryptography and AES in the docker container when such files are encountered but it is nothing to worry about, they will be simply ignored and the system will continue to function normally. - External storages (through `files_external`) may not work as well as the local storage. - Multi-tenant installation is not supported. You can only run one context chat instance connected to one Nextcloud instance (see [the AppAPI FAQ](/link/985) ). # App: Context Agent (context_agent) The *context_agent* app is the app that provides AI agent functionality in Nextcloud's "Chat with AI" feature and acts as a backend for the [Nextcloud Assistant app](/link/993). Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities. When the Context Agent app is installed the AI Chat in Nextcloud Assistant will be able to interact with your Nextcloud apps via virtual integrations that are called "tools". They allow the Assistant to perform actions in your Nextcloud upon sending instructions in a chat message. Tool groups are only available if their requirements are met. They can be en- and disabled in the AI admin settings. Additionally, the app makes all tools available via an MCP server that you can connect third-party agents to (see below). ## Currently implemented tools ### Artificial intelligence tools - Ask a question to context chat (requires [Context Chat](/link/999)) - Example prompt: *"What is the company's sick leave process?"* - Transcribe a media file (requires Transcribe audio task type enabled) - Example prompt: *"Can you transcribe the following file? https://mycloud.com/f/9825679"* (Can be selected via smart picker.) - Generate documents (requires [Nextcloud Office](https://apps.nextcloud.com/apps/richdocuments)) - Example prompt: *"Can you generate me a slide deck for my presentation about cats?"* - Example prompt: *"Can you generate me a spreadsheet with some plausible numbers for countries and their population count?"* - Example prompt: *"Can you generate me a pdf with an outline about what to see in Berlin?"* - Generate images (requires Image generation task type enabled) - Example prompt: *"Can you generate me an image of a cartoon drawing of a roman soldier typing something on a laptop?"* ### Calendar tools - List the user's calendars - Example prompt: *"List my calendars"* - Schedule an event in the user's calendar - Example prompt: *"Schedule an event with Andrew tomorrow at noon."* - Find free times in users' calendar - Example prompt: *"Find a free 1-hour slot for a meeting with me and Marco next week."* ### Tasks tools - Create a task - Example prompt: *"Create a task for grocery shopping with due date tomorrow."* - List tasks - Example prompt: *"List my outstanding tasks"* - Complete a task - Example prompt: *"Mark the grocery shopping task as completed."* - Update a task's details - Example prompt: *"Change the priority of the grocery shopping task to the highest possible priority."* - Example prompt: *"Change the due date of my work report task to the beginning of next week."* - Delete a task - Example prompt: *"Delete the grocery shopping task in my tasks."* ### Circles/teams tools - List circles - Example prompt: *"List all my teams."* - List circle members - Example prompt: *"List all members of my Content Marketing team."* - Create new circle - Example prompt: *"Create a new team called 'Hiking group'."* - Add members to circles - Example prompt: *"Add Ralph to the Hiking group team."* - Remove members from circles - Example prompt: *"Remove ralph from the Hiking group team."* - Change circle details - Example prompt: *"Change the name of the Hiking group team to 'Outdoor group'."* - Example prompt: *"Add the following description to the Hiking group team: We go hiking together once a month. Come join us."* - Delete a circle - Example prompt: *"Delete the Hiking group team."* - Share a file with a circle - Example prompt: *"Share my Hiking plans.md file with the Hiking group team."* ### Collectives tools (require [Collectives](https://apps.nextcloud.com/apps/collectives)) - List the user's collectives - Example prompt: *"List my collectives."* - List the pages in a collective - Example prompt: *"List the pages in my Engineering collective."* - Get metadata about a page in a collective - Example prompt: *"Show me the details of the 'Onboarding' page in my Engineering collective."* - Get the markdown content of a page - Example prompt: *"Fetch the content of the 'Onboarding' page in my Engineering collective."* - List trashed pages in a collective - Example prompt: *"List the trashed pages in my Engineering collective."* - Create a new page in a collective under a given parent - Example prompt: *"Create a new 'Q3 retro' page in my Engineering collective under the 'Retros' page."* - Replace the content of a page (a note saying the page was edited with the help of the AI Assistant is appended automatically) - Example prompt: *"Update the 'Q3 retro' page in my Engineering collective with the meeting notes."* - Rename a page - Example prompt: *"Rename the 'Q3 retro' page in my Engineering collective to 'Q3 2026 retro'."* - Move a page to a different parent - Example prompt: *"Move the 'Q3 2026 retro' page under the 'Archive' page in my Engineering collective."* - Set the emoji icon of a page - Example prompt: *"Set the emoji of the 'Onboarding' page in my Engineering collective to 🚀."* - Move a page to the collective's trash - Example prompt: *"Trash the 'Old draft' page in my Engineering collective."* - Restore a page from the collective's trash - Example prompt: *"Restore the 'Old draft' page from the trash of my Engineering collective."* - Permanently delete a page - Example prompt: *"Permanently delete the 'Old draft' page in my Engineering collective."* ### Contacts tools - Find a contact - Example prompt: *"What is Anna's email address?"* - Find a user's ID - Example prompt: *"What is Ralph's userID?"* - Find the current user's details - Example prompt: *"Where do I live?"* ### Cookbook tools - List recipes - Example prompt: *"List my recipes."* - Search for recipes - Example prompt: *"Do I have any Spaghetti recipes?"* - Get recipe details - Example prompt: *"Can you give me the details of my Spaghetti Carbonara recipe?"* - Create a new recipe - Example prompt: *"Create a recipe for Guacamole in my cookbook."* - Delete a recipe - Example prompt: *"Remove the Guacamole recipe from my cookbook."* - List recipe categories - Example prompt: *"Which recipe categories do I have in my cookbook?"* ### Deck tools (require [Deck](https://apps.nextcloud.com/apps/deck)) - List deck boards - Example prompt: *"List the deck boards I have access to."* - List the cards on a deck board - Example prompt: *"List the cards on my Personal deck board."* - Add a new card - Example prompt: *"Can you add a card with title 'Repair kitchen sink' to my Personal deck board?"* - Add a label to a card - Example prompt: *"Can you add the label 'Urgent' to the 'repair kitchen sink' card in my personal deck board?"* - Assign a card to a user - Example prompt: *"Can you assign the 'Repair kitchen sink' card in my Personal deck board to Andrew?"* - Delete a card - Example prompt: *"Delete the 'Repair kitchen sink' card in my Personal deck board."* - List the comments on a deck card - Example prompt: *"Show the comments on the 'Repair kitchen sink' card in my Personal deck board."* - Add a comment to a deck card - Example prompt: *"Add a comment 'I'll handle this Friday' to 'Repair kitchen sink' in my Personal deck board."* - Edit a comment on a deck card - Example prompt: *"Update my last comment on the 'Repair kitchen sink' card to say 'Moved to Saturday'."* - Delete a comment on a deck card - Example prompt: *"Delete my last comment on the 'Repair kitchen sink' card in my Personal deck board."* ### Files tools - Get contents of a file - Example prompt: *"Can you fetch the following file in my documents? Design/Planning.md"* - Example prompt: *"Can you fetch the following file in my documents? https://mycloud.com/f/98543234"* - Retrieve folder tree - Example prompt: *"List my files."* - Create a public link for a file or folder - Example prompt: *"Create a public link for the following file: Design/Planning.md"* - Create a new file - Example prompt: *"Create a new file Ideas.md in my files and fill it with ideas for hiking destinations in the black forest."* - Create a new folder - Example prompt: *"Create a new folder 'Hiking plans' in my files."* - Move a file - Example prompt: *"Move the Ideas.md file into the Hiking plans folder."* - Copy a file - Example prompt: *"Copy the Ideas.md file into my Notes folder."* - Delete a file - Example prompt: *"Delete the Ideas.md file."* ### Forms tools - List all forms - Example prompt: *"List all the forms I have access to."* - Get details of a form - Example prompt: *"Can you give me all details about the Retreat signup form?"* - Add a question to a form - Example prompt: *"Add the following question to the retreat signup form: 'Number of days attending'."* - Retrieve all responses of a form - Example prompt: *"List all responses to the Retreat signup form."* - Update form settings - Example prompt: *"Make the Retreat signup form expire end of next week."* - Delete a form - Example prompt: *"Delete the Retreat signup form."* ### Bookmarks tools - List all bookmarks - Example prompt: *"List all my bookmarks."* - Add a bookmark - Example prompt: *"Add a bookmark for https://nextcloud.com with title 'Nextcloud homepage'."* - Delete a bookmark - Example prompt: *"Delete the bookmark for https://nextcloud.com."* - Update a bookmark - Example prompt: *"Change the title of the bookmark for https://nextcloud.com to 'Nextcloud official homepage'."* - Example prompt: *"Add the tag 'cloud' to the bookmark for https://nextcloud.com."* - Example prompt: *"Remove the tag 'cloud' from the bookmark for https://nextcloud.com."* - Example prompt: *"Put the bookmark for https://nextcloud.com into the 'work' folder."* - List bookmark folders - Example prompt: *"Which bookmark folders do I have?"* - Create a bookmark folder - Example prompt: *"Create a bookmark folder called 'work'."* - List bookmark tags - Example prompt: *"Which bookmark tags do I have?"* ### Search tools All search providers in Nextcloud are also automatically available as tools. - Search for files - Example prompt: *"List all the powerpoint presentations in my files with file ending pptx."* ### Share tools - List shares - Example prompt: *"List all files that were shared with me."* - Example prompt: *"List the shares of the Design/Ideas.md file."* - Share a file or folder with a user - Example prompt: *"Share the Design/Ideas.md file with the user martin."* - Share a file or folder with a group - Example prompt: *"Share the Design/Ideas.md file with the group Designers."* - Update a share's permissions - Example prompt: *"Only allow martin read only access on the share of the Design/Ideas.md file."* - Delete a share - Example prompt: *"Remove the share of the Design/Ideas.md file with martin."* - List user groups - Example prompt: *"Which user groups are there?"* - Retrieve share details > - Example prompt: *"Does martin have write access to the Design/Ideas.md file I shared with him?"* ### Talk tools (require [Talk](https://apps.nextcloud.com/apps/spreed)) - List the user's talk conversations - Example prompt: *"List my talk conversations"* - List messages in a talk conversation - Example prompt: *"List the latest messages in my conversation with Andrew"* - Send a message to a talk conversation - Example prompt: *"Can you send a joke to Andrew in talk?"* - Create a public talk conversation - Example prompt: *"Can you create a new public talk conversation titled 'Press conference'?"* - Reply to a specific message in a talk conversation - Example prompt: *"Reply to Andrew's last message in our talk conversation with 'Got it, thanks!'"* - Add an emoji reaction to a message in a talk conversation - Example prompt: *"React with 👍 to Andrew's last message in our talk conversation"* - Remove an emoji reaction from a message in a talk conversation - Example prompt: *"Remove my 👍 reaction from Andrew's last message"* - List the reactions on a message in a talk conversation - Example prompt: *"Who reacted to Andrew's last message in our talk conversation?"* - Create a poll in a talk conversation - Example prompt: *"Create a poll in the 'Team standup' conversation asking 'Which day works for the offsite?' with options Monday, Tuesday, Wednesday"* - Get the question, options, and current results of a poll in a talk conversation - Example prompt: *"Show the current results of the offsite poll in the 'Team standup' conversation"* - Cast a vote on a poll in a talk conversation - Example prompt: *"Vote for Tuesday on the offsite poll in the 'Team standup' conversation"* - Close a poll in a talk conversation - Example prompt: *"Close the offsite poll in the 'Team standup' conversation"* - Share a Nextcloud Files item to a talk conversation - Example prompt: *"Share the file 'Q3 plan.pdf' to my conversation with Andrew"* - List items of a given type (e.g. file, location, poll) that were shared in a talk conversation - Example prompt: *"List the files shared in my conversation with Andrew"* - Get an overview of items shared in a talk conversation across all types - Example prompt: *"Give me an overview of what's been shared in my conversation with Andrew"* ### Mail tools (require [Mail](https://apps.nextcloud.com/apps/mail)) - Send an email via Nextcloud Mail - Example prompt: *"Send a test email from my carry@company.com account to Andrew@company.com"* - List all connected mail accounts - Example prompt: *"List my mail accounts"* - List all mail folders of an email account - Example prompt: *"List the folders of my carry@company.com account"* - List the mails in a mail folder - Example prompt: *"List the last 5 mails in the inbox of my carry@company.com account"* ### Miscellaneous tools - Get coordinates for an Address from Open Street Maps Nomatim - Example prompt: *"What are the coordinates for Berlin, Germany?"* - Get the URL for a map of a location using Open Street Maps - Example prompt: *"Can you show me a map of New York, please"* - Get the current weather at a location - Example prompt: *"How is the weather in Berlin?"* - Search for youtube videos - Example prompt: *"Show me the youtube video of the Nextcloud hub 10 launch."* - Search Duckduckgo - Example prompt: *"Show me search results for quick pasta recipes, please."* - Determine public transport routes (requires a [HERE](https://www.here.com/) API key configured in the admin settings) - Example prompt: *"How can I get from Würzburg Hauptbahnhof to Berlin Hauptbahnhof?"* - List all projects in OpenProject (requires the [OpenProject integration](https://apps.nextcloud.com/apps/integration_openproject)) - Example prompt: *"List all my projects in OpenProject, please"* - List all available assignees of a project in OpenProject (requires the [OpenProject integration](https://apps.nextcloud.com/apps/integration_openproject)) - Example prompt: *"List all available assignees for the 'Product launch' project in OpenProject"* - Create a new work package in a given project in OpenProject (requires the [OpenProject integration](https://apps.nextcloud.com/apps/integration_openproject)) - Example prompt: *"Create a work package called 'Publish release video' in the 'Product launch' project in OpenProject"* ## Combining tools These tools can also be combined by the agent to fulfil tasks like the following: - *"How is the weather where Andrew lives?"* - Uses contacts to look up Andrew's address and then checks the weather - *"How is the weather where I live?"* - Look up the current user's address and then checks the weather - *"Send an email from carry@company.com to Andrew"* - Uses contacts to look up Andrew's email and then sends an email - *"Which of my files are from Anna?"* - Looks up Anna's userID and searches for files that belong to her - *"Send the content of my draft.md file to Andrew in Talk"* - Gets the content of the file and sends it in a 1-1 Talk conversation with Andrew ## Custom Tools using MCP Model Context Protocol (MCP) is a protocol that enables Large Language Models (LLMs) to interact with external data sources and tools. The Context Agent app allows administrators to extend its capabilities by adding custom services via MCP. This can be configured in the admin settings under "MCP Config," where you can provide a JSON configuration in the following format: ``` json { "service-name": { "url": "https://service-url.com/endpoint", "transport": "streamable_http" } } ``` ## Requirements - This app is built as an External App and thus depends on AppAPI v3.1.0 or higher - Nextcloud AIO is supported - No GPU is necessary for Context Agent but one might be useful if you use it with a self-hosted provider like llm2 - CPU Sizing > - At least 1GB of system RAM ## Installation 0. Make sure the [Nextcloud Assistant app](/link/993) is installed 1. [Install AppAPI and setup a Deploy Demon](/link/985) 2. Install the "Context Agent" ExApp via the "Apps" page in the Nextcloud web admin user interface 3. If you want to use the agent in the Nextcloud UI, install a text generation backend like [llm2](/link/995) or [integration_openai](/link/1005) via the "Apps" page in Nextcloud; if you only want to use the MCP server you don't need these. ### Model requirements This app requires underlying Large language models to support tool calling. The default model in *llm2* does support tool calling since version 2.4.0. Other models that may give good results are: - Google Gemma 3 12B or higher - Mistral 3 small 24B - Qwen 2.5 8B or higher (May not work well with languages other than English) See [llm2 documentation](/link/995) on how to configure alternate models. ## Using Nextcloud MCP Server Context Agent exposes an MCP server that can be used by other large language models or applications to access the tools provided by Context Agent. The server will be available at https://your-nextcloud-domain.com/index.php/apps/app_api/proxy/context_agent/mcp/, and it requires authentication via an app password passed in the Authorization header. Ex: Authorization: Bearer \. ## Scaling It is currently not possible to scale this app, we are working on this. ## App store You can also find the app in our app store, where you can write a review: ## Repository You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: Nextcloud customers should file bugs directly with our Support system. ## Known Limitations - Make sure to test the language model you are using in concert with this app for whether they meet the use-case's quality requirements - Most models have difficulties with languages other than English. Some models sometimes answer in a different language than the one used by the user. - Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI). We still try to optimize this as far as possible, so if you encounter any false or problematic output, you can report it [in a dedicated Github issue](https://github.com/nextcloud/context_agent/issues/51) to help us improve this app. - When multiple MCP services are configured that have tools with the same name undefined behavior will occur. - Only remote MCP services are supported (streamable_http transport). - MCP services that require different access tokens for each user are not currently supported. # App: Summary Bot (Talk chat summarize bot) The *Summary Bot* app utilizes Large Language Model (LLM) providers in Nextcloud and can be added to a conversation in Nextcloud Talk to generate summaries from the chat messages of that room either on-demand or following a schedule. It can run on only open source or proprietary models either on-premises or in the cloud leveraging apps like [Local large language model app](https://apps.nextcloud.com/apps/llm2) or [OpenAI and LocalAI integration app](https://apps.nextcloud.com/apps/integration_openai). Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities. The app currently supports the following languages: - English (en) The quality of summaries depends directly on the quality of the underlying model. It is recommended to test the model for the desired use-case before applying it. ## Requirements - Minimal Nextcloud version: 30 - Docker - AppAPI \>= 3.0.0 - Talk - Task Processing Provider like Local large language model app (llm2) or OpenAI and LocalAI integration app (integration_openai) ### Space usage - ~100MB ## Installation 0. Make sure the following apps are installed: - [Nextcloud AppAPI app](https://apps.nextcloud.com/apps/app_api) - [Nextcloud Talk app (Spreed)](https://apps.nextcloud.com/apps/spreed) - One of the following AI model providers: - [Nextcloud Local large language model app](https://apps.nextcloud.com/apps/llm2) - [Nextcloud OpenAI and LocalAI integration app](https://apps.nextcloud.com/apps/integration_openai) - [Nextcloud IBM watsonx.ai integration app](https://apps.nextcloud.com/apps/integration_watsonx) ### Setup (via App Store) 1. Install the *Summary Bot* app via the "Apps" page in Nextcloud 2. Enable the *Summary Bot* Bot for the selected Chatroom via the three dots menu of the Chatroom (The Bots settings are located inside the *Bots* section) ### Setup (Manual) After cloning this app *manually* (cloned via git to your apps directory) you will need to execute the following steps: 1\. Change to the folder you have cloned the source to: .. code-block: cd /path/to/your/nextcloud/webroot/apps/summary_bot/ 2\. Build the docker image: .. code-block: docker build --no-cache -f Dockerfile -t local_summary_bot . 3. Run the docker image: *Info:* - APP_VERSION environment variable should be equal to the version of the *Summary Bot* you are using - NEXTCLOUD_URL environment variable must be set to your Nextcloud instance's URL, ensuring it's reachable by the docker image. sudo docker run -ti -v /etc/localtime:/etc/localtime:ro -v /etc/timezone:/etc/timezone:ro -e APP_ID=summary_bot -e APP_DISPLAY_NAME="Summary Bot" -e APP_HOST=0.0.0.0 -e APP_PORT=9031 -e APP_SECRET=12345 -e APP_VERSION=1.0.0 -e NEXTCLOUD_URL='' -p 9031:9031 local_summary_bot 4. Un-register the Summary Bot if its already installed sudo -E -u www-data php occ app_api:app:unregister summary_bot 5. Register the Summary Bot so that your Nextcloud instance is aware of it *Info:* Adjust the host value in the following example to the IP address of the docker container (for added security) sudo -E -u www-data php occ app_api:app:register summary_bot manual_install --json-info '{ "id": "summary_bot", "name": "Summary Bot", "daemon_config_name": "manual_install", "version": "1.0.0", "secret": "12345", "host": "0.0.0.0", "port": 9031, "scopes": ["AI_PROVIDERS", "TALK", "TALK_BOT"], "protocol": "http"}' --force-scopes --wait-finish 6. Enable the *Summary Bot* for the selected Chatroom via the three dots menu of the Chatroom (The Bots settings are located inside the *Bots* section) ## Usage After enabling the *Summary Bot* in a Chatroom, you can test its functionality by simply sending the message below: > "@summary" or "@summary help" ## App store You can also find the app in our app store, where you can write a review: ## Repository You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: Nextcloud customers should file bugs directly with our Customer Support. ## Ethical AI Rating The ethical rating of the *Summary Bot*, which utilizes a model for text processing through the Nextcloud Assistant app, is significantly influenced by the choice and implementation of the underlying model. Learn more about the Nextcloud Ethical AI Rating [in our blog](https://nextcloud.com/blog/nextcloud-ethical-ai-rating/). ## Known Limitations - The Summary Bot cannot access previous conversations, it only recognizes messages from the moment it was enabled in the chatroom. - Summary of maximum 40000 characters is supported. This assumes the underlying model can handle this amount of text (which should be close to 16000 context length). - Languages other than English are not supported. The underlying model may still be able to understand other languages. - AI models may occasionally produce inaccurate information. Therefore, they should be employed with caution in non-critical scenarios. It's essential to verify the accuracy of the bot's output before application. - Be aware that AI models can consume a significant amount of energy. It's advisable to consider this factor in the planning and operation of AI systems if hosted on-premises or sustainability is a concern. - AI models can exhibit extended processing times when run on CPUs. For enhanced efficiency, utilizing GPU support is recommended to expedite request handling. - Customer support is available upon request, however we can't solve false or problematic output (hallucinations), most performance issues, or other problems caused by the underlying models. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI) # App: Local Text-To-Speech (text2speech_kokoro) The *text2speech_kokoro* app is one of the apps that provide Text-To-Speech functionality in Nextcloud and act as a speech generation backend for the [Nextcloud Assistant app](/link/993) and other apps making use of the core Text-To-Speech Task type\\`. The *text2speech_kokoro* app specifically runs only open source models and does so entirely on-premises. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities. This app uses [Kokoro](https://github.com/hexgrad/kokoro) under the hood. The used model supports the following languages: - American English - British English - Spanish - French - Italian - Hindi - Portuguese - Japanese - Mandarin ## Requirements - Minimal Nextcloud version: 31 - This app is built as an External App and thus depends on AppAPI v2.3.0 - Nextcloud AIO is supported - We currently support x86_64 CPUs - We do not support GPUs - CPU Sizing > - The more cores you have and the more powerful the CPU the better, we recommend around 10 cores > - The app will hog all cores by default, so it is usually better to run it on a separate machine > - 800MB RAM ## Installation 0. Make sure the [Nextcloud Assistant app](/link/993) is installed 1. [Install AppAPI and setup a Deploy Demon](/link/985) 2. Install the *text2speech_kokoro* "Local Text-To-Speech" ExApp via the "Apps" page in the Nextcloud web admin user interface ## Scaling It is currently not possible to scale this app, we are working on this. Based on our calculations an instance has a rough capacity of 4h of transcription throughput per minute (measured with 8 CPU threads on an Intel(R) Xeon(R) Gold 6226R). It is unclear how close to real-world usage this number is, so we do appreciate real-world feedback on this. ## App store You can also find this app in our app store, where you can write a review: ## Repository You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: Nextcloud customers should file bugs directly with our customer support. ## Known Limitations - We currently only support languages supported by the underlying Kokoro model - The Kokoro models perform unevenly across languages, and may show lower accuracy on low-resource and/or low-discoverability languages or languages where there was less training data available. - Make sure to test the language model you are using it for whether it meets the use-case's quality requirements - Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI) # App: Live Transcription and Translation in Nextcloud Talk (live_transcription) This app provides live transcription and translation of speech in Nextcloud Talk calls using open source AI models provided by [Vosk](https://alphacephei.com/vosk/).\ The transcription is done on your own server, preserving your privacy and data sovereignty, while the translation is done using a translation task processing provider like the [translate2 app](/link/994). [OpenAI and LocalAI integration](https://apps.nextcloud.com/apps/integration_openai) and [DeepL integration](http://apps.nextcloud.com/apps/integration_deepl) apps will soon also be supported for translation. A good set of language models for transcription are auto-downloaded. They include Arabic, Arabic (Tunisian), Breton, Catalan, Czech, German, English, Esperanto, Spanish, Persian (Farsi), French, Hindi, Italian, Japanese, Kazakh, Korean, Dutch, Polish, Portuguese (Brazilian), Russian, Telegu, Tajik, Turkish, Ukrainian, Uzbek, Vietnamese and Chinese.\ The translation capabilities depend on the installed translation task processing provider app. A list of translation-capable apps can be found [here](/link/992) in the "Backend apps" section. ## Installation 1. Make sure the [Nextcloud Talk app](https://apps.nextcloud.com/apps/spreed) is installed. 2. Make sure the High-Performance Backend (latest or released after September 2025) is installed and configured in Nextcloud Talk settings. See the [Nextcloud Talk install manual](https://nextcloud-talk.readthedocs.io/en/latest/quick-install/) for more information. 3. Setup a [Deploy Daemon](/link/985) in AppAPI Admin settings. 4. Install the **live_transcription** app via the "Apps" page in Nextcloud, or by executing occ app_api:app:register live_transcription \ --env LT_HPB_URL=wss://cloud.example.com/standalone-signaling/spreed \ --env LT_INTERNAL_SECRET=1234 \ --wait-finish > **说明** > > The environment variables `LT_HPB_URL` and `LT_INTERNAL_SECRET` must be set in the [Deploy Options](/link/990) during installation, and the High-Performance Backend must be functionally configured in Nextcloud Talk settings for the app to work. > > Changing these environment variables after installation is possible through a re-installation of the app after uninstalling it first. 5. Install a Text-to-text task processing provider app for translation capabilities from the "Backend apps" section [here](/link/992). ## Requirements - Minimal Nextcloud version: 33 - Nextcloud AIO is supported - We currently support NVIDIA GPUs and x86_64 CPUs. Only CPU-based transcription is also supported and works well on modern x86 CPUs. - CUDA \>= v12.4.1 on your host system for GPU-based transcription - GPU Sizing > - A NVIDIA GPU with at least 10 GB VRAM > - 16 GB of system RAM should be enough for one or two concurrent calls - CPU Sizing > - x86 CPU with 4 threads. Additional 2 threads per concurrent call. > - 16 GB of RAM should be enough for one or two concurrent calls - Space usage - ~ 2.8 GB for the docker container - ~ 6.0 GB for the default models > **提示** > > We currently have very little real-world experience running this software on production instances. The above sizing recommendations come from our estimates and are not real-world benchmarks. Actual requirements will vary based on factors such as the number of concurrent calls, audio quality, and selected languages. Please do thorough testing to confirm your hardware meets your needs. ## Logs Warnings and errors are logged to the main Nextcloud server logs.\ To inspect the detailed JSON logs for lower levels, the following commands can help. To view docker stdout/err logs: ``` bash docker logs -f -n400 nc_app_live_transcription ``` To view the JSON logs:\ The logs are rotated when file size exceeds 20 MiB. The older log files are named `lt.log.1`, `lt.log.2`, and so on. ``` bash docker exec -it nc_app_live_transcription tail -f -n400 /nc_app_live_transcription_data/logs/lt.log ``` To download the JSON logs: ``` bash docker cp nc_app_live_transcription:/nc_app_live_transcription_data/logs/ /tmp/nc_app_live_transcription_logs ``` ## App store You can also find the app in our app store, where you can write a review: ## Repository You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: Nextcloud customers should file bugs directly with our Customer Support. ## Limitations - The generated transcripts may not be perfect and may contain errors. It can also depend on the audio quality and the speaker's accent. - The app currently supports only a limited number of languages. More languages may be added in the future. - The languages other than English may have lower accuracy mainly due to the shipped models being smaller. - The app currently does not support punctuation in the transcription. - [OpenAI and LocalAI integration](https://apps.nextcloud.com/apps/integration_openai) and [DeepL integration](http://apps.nextcloud.com/apps/integration_deepl) apps are not yet supported for translation. # App: Local Optical Character Recognition (ocr_paddle) The *ocr_paddle* app is one of the apps that provide Optical Character Recognition functionality in Nextcloud and act as an OCR backend for the [Nextcloud Assistant app](/link/993) and [other apps making use of the core OpticalCharacterRecognition Task type](/link/992). The *ocr_paddle* app specifically runs only open source models and does so entirely on-premises. Nextcloud can provide customer support upon request, please talk to your account manager for the possibilities. This app uses [PaddleOCR_VL](https://huggingface.co/PaddlePaddle/PaddleOCR-VL) under the hood. The used model supports 109 languages, covering major global languages, including but not limited to Chinese, English, Japanese, Latin, and Korean, as well as languages with different scripts and structures, such as Russian (Cyrillic script), Arabic, Hindi (Devanagari script), and Thai. PDFs are supported; multi-page PDFs come back as one document. ## Requirements - Minimal Nextcloud version: 35 - This app is built as an External App and thus depends on at least AppAPI v3.1.0 - Nextcloud AIO is supported - We currently support x86_64 & arm64 CPUs as well as NVIDIA GPUs - CPU Sizing > - If you don't have a GPU, this app will utilize your CPU cores > - The more cores you have and the more powerful the CPU the better, we recommend 10-20 cores, but the runtime will still be very slow on CPU only > - The app will hog all cores by default, so it is usually better to run it on a separate machine > - 7.5GB of free system RAM - GPU Sizing > - NVIDIA GPU > - Minimum of 6GB free VRAM ## Installation 0. Make sure the [Nextcloud Assistant app](/link/993) is installed 1. [Install AppAPI and setup a Deploy Daemon](/link/985) 2. Install the *ocr_paddle* "Local OCR: PaddleOCR" ExApp via the "Apps" page in the Nextcloud web admin user interface ## Scaling It is currently not possible to scale this app, we are working on this. If you have real world data on how many files this app can process per minute, we will gladly list it here. ## App store You can also find this app in our app store, where you can write a review: ## Repository You can find the app's code repository on GitHub where you can report bugs and contribute fixes and features: Nextcloud customers should file bugs directly with our customer support. ## Known Limitations - We currently only support languages supported by the underlying PaddleOCR-VL model - The PaddleOCR-VL model performs unevenly across languages, and may show lower accuracy on low-resource and/or low-discoverability languages or languages where there was less training data available. - Only the first 50 pages of PDF files are read (configurable via the `OCR_MAX_PDF_PAGES` deploy option), longer PDFs are silently truncated. - PDF pages are rasterized at 170 DPI (configurable via the `OCR_PDF_DPI` deploy option) before OCR - Inputs are downscaled to ≤2.8 MP (max_pixels: 2822400), so high-resolution scans lose detail before OCR runs. - Make sure to test the language model you are using it for whether it meets the use-case's quality requirements - Customer support is available upon request, however we can't solve false or problematic output, most performance issues, or other problems caused by the underlying model. Support is thus limited only to bugs directly caused by the implementation of the app (connectors, API, front-end, AppAPI) # AI as a Service At Nextcloud, we focus on creating on-premise AI apps that run fully self-hosted on your own servers in order to preserve your privacy and data sovereignty. However, you can also offload these resource-heavy tasks to an "AI as a Service" provider offering API access in exchange for payment. Examples of such providers are [OpenAI](https://platform.openai.com/), with its ChatGPT APIs providing language model access among other APIs, as well as [Replicate](https://replicate.com/) and [IBM watsonx](https://www.ibm.com/watsonx). ## Installation In order to use these providers you will need to install the respective app from the app store: - `integration_openai` - `integration_replicate` - `integration_watsonx` You can then add your account information, set rate limits, and set the providers live in the "Artificial intelligence" section of the administration settings. Optionally (but recommended), setup background workers for faster pickup of tasks. See [the relevant section in AI Overview](/link/992) for more information. ## OpenAI integration With this application, you can also connect to a self-hosted LocalAI or Ollama instance or to any service that implements an API similar enough to the OpenAI API, for example [IONOS AI Model Hub](https://docs.ionos.com/cloud/ai/ai-model-hub), [Plusserver](https://www.plusserver.com/en/ai-platform/), [Groqcloud](https://console.groq.com), [MistralAI](https://mistral.ai) or [Together AI](https://together.ai). Do note, however, that we test the Assistant tasks that this app implements only with OpenAI models and only against the OpenAI API, we thus cannot guarantee other models and APIs will work. Some APIs claiming to be compatible with OpenAI might not be fully compatible so we cannot guarantee that they will work with this app. ## IBM watsonx.ai integration With this application, you can also connect to a self-hosted cluster running the IBM watsonx.ai software. Do note, however, that we test the Assistant tasks that this app implements only with the provided foundation models and only against IBM Cloud servers. We thus cannot guarantee that other models or server instances will work. ## Improve performance Prompts from these apps can have a delay of up to 5 minutes. This can be optimized and more information can be found in [the relevant section in AI Overview](/link/992). # Insight and debugging In order to gain insights and debug AI tasks, there are a number of occ commands available in the taskprocessing namespace. All commands listed here accept an --output parameter that can be set to plain, json or json_pretty. Tasks are retained in the database for 6 months. Note, however, that there is no strict write-only guarantee for these logs. This means Nextcloud will not change the task logs in the database once the task has ended, but any administrator with database access has the power to change them. ## Get a task by id $ occ taskprocessing:task:get For example $ occ taskprocessing:task:get --output=json_pretty 42 { "id": 1, "type": "core:text2text:chat", "lastUpdated": 1759739466, "status": "STATUS_SUCCESSFUL", "userId": "admin", "appId": "assistant:chatty-llm", "input": { "system_prompt": "This is a conversation in a specific language between the user and you, Nextcloud Assistant. You are a kind, polite and helpful AI that helps the user to the best of its abilities. If you do not understand something, you will ask for clarification. Detect the language that the user is using. Make sure to use the same language in your response. Do not mention the language explicitly.", "input": "What's the weather in Berlin today?", "history": [] }, "output": { "output": "I'm happy to help, but I'm a large language model, I don't have real-time access to current weather conditions. However, I can suggest checking a reliable weather website or app, such as AccuWeather or OpenWeatherMap, for the most up-to-date information on the weather in Karlsruhe today. Would you like me to help with anything else?" }, "customId": "chatty-llm:1", "completionExpectedAt": 1759739513, "progress": 1, "scheduledAt": 1759739453, "startedAt": 1759739455, "endedAt": 1759739466, "allowCleanup": true, "error_message": null } Each task has the following fields: > - id The internal ID of the task, also referenced in user-facing error messages in case something goes wrong > - type The type of the task > - status The current status of the task (can be either "STATUS_CANCELLED", "STATUS_FAILED", "STATUS_SUCCESSFUL", "STATUS_SCHEDULED", "STATUS_RUNNING", or "STATUS_UNKNOWN") > - userId The Id of the user who requested the task > - lastUpdated When the task was last updated > - scheduledAt When the task was scheduled/created > - startedAt When the task was started to be processed by the set task processing provider > - completionExpectedAt When the system expects/expected the task to be finished > - endedAt When the task finished be it successfully or unsuccessfully > - appid The ID of the app that scheduled the task > - input The values that were part of the task input > - output The values that were part of the task output > - error_message The error message in case the task failed ## List and filter tasks $ occ taskprocessing:task:list [options] -u, --userIdFilter[=USERIDFILTER] only get the tasks for one user ID -t, --type[=TYPE] only get the tasks for one task type --appId[=APPID] only get the tasks for one app ID --customID[=CUSTOMID] only get the tasks for one custom ID -s, --status[=STATUS] only get the tests that have a specific status --scheduledAfter[=SCHEDULEDAFTER] only get the tasks that were scheduled after a specific date (Unix timestamp) --endedBefore[=ENDEDBEFORE] only get the tasks that ended before a specific date (Unix timestamp) For example $ occ taskprocessing:task:list --output=json_pretty --status=3 --scheduledAfter=1759740266 --endedBefore=1759743900 [ { ... } ] # Legal: Compliance with EU AI Act ## Implementation of the transparency requirements This section describes of how Nextcloud and its AI products implemented the transparency requirements. - All functionality that outputs AI generated content which significantly altered the user’s input has in the software UI a visual warning that the content is generated using AI and urge users to double check the correctness of any claims therein. - Additionally, AI generated files like documents, images, and audio contain a note that it was generated by AI. We also add a machine-readable tag (“Generated using AI”). In file formats that support metadata, we add metadata with the same information (“Generated using Artificial Intelligence.”). - Agentic interactions with third parties always include a note that the interaction was generated by AI (e.g. E-Mails sent on behalf of the user, calendar events created on behalf of the user, etc.). - Employees of Nextcloud GmbH have been instructed through the internal AI policy to inform audience when they send or publish content that were generated by AI or significantly altered by AI. - All interactions of users with AI in Nextcloud are retained in the database for observability and transparency. Refer to [Insight and Debugging](/link/1006) for details on how to explore and examine these records. - All changes to AI configuration will be logged in the [audit log](/link/932). ## Reliability and robustness This section describes the measures that ensure the software is reliable and robust. - All merged code is reviewed by at least one extra employee who checks for potential issues. - The design and implementation of larger and more risky new features are always discussed with multiple experts to ensure a robust implementation. - Our software is entirely open-source and anyone can raise bugs they face and propose fixes to the bugs they face. The incoming bugs are regularly reviewed by our employees and prioritized accordingly. Bugs that affect the functionality of the software for many users and instances are prioritized high and the work to develop a fix is added to our roadmap. For the other bugs, anyone is welcome to propose a fix and our employees will review the code change. - We offer an enterprise subscription for downstream providers with critical infrastructure. This will guarantee support and fixes for any issue they face within a SLA agreement. - We regularly test our software using standard testing procedures, like static code analysis and integration tests where appropriate. - We maintain several test instances to ensure the reliability and stability of our AI features. One instance is updated daily with the latest development versions and a selection of self-hosted AI features are tested here. Another instance is used to validate upcoming core releases prior to announcement, where we test a selection of AI features of which most depend on OpenAI as a backend. In addition, we operate an instance that resembles the production environment of a small to mid-sized company, where we perform end-to-end testing of selected AI features. This environment combines AI-as-a-service providers for text generation capabilities with self-hosted models for other capabilities, allowing us to verify real-world performance and usability. - When a feature relies on large language models as its core component, we cannot guarantee complete reliability due to the unpredictable nature of an LLM, and have thus documented the limitations of said features and provide AI literacy training to our employees and customers. We select the LLM models we recommend to use based their results on industry standard benchmarks as well as on a custom suite of tests for multilingual usage and tool calling. In the user-facing UI we prompt users to always double check the AI generated content. ## Interoperability This section describes the measures that ensure the software is interoperable. - We provide our AI features via an open API with publicly accessible [OpenAPI specs](https://docs.nextcloud.com/server/latest/developer_manual/_static/openapi.html#/) which allows developers to build on top of our features. - As our software is fully open-source, anyone can adjust the software to meet their needs. For example, anyone can adjust the core code, adjust the code of existing applications, or develop a custom application for Nextcloud. - We implement integrations for the major model hosting providers and their protocols upon request of customers. We are interoperable with OpenAI and IBM watsonx. As Nextcloud is an open app ecosystem, anyone can develop an integration with a model hosting provider on their own. - We implement the agent interoperability protocol MCP both as a client and as a server to allow users to connect the AI Agent software to existing services and connect existing AI Agents to our software. - We implement a local model hosting mechanism that can be used to host GGUF models (most open weight models can be converted using an Open Source tool called llama.cpp). ## Cybersecurity and physical security of the hardware This section describes the measures how we guarantee cybersecurity and the physical security of the hardware. - All merged code is reviewed by at least one employee who also checks on security. - We have a security bounty program at HackerOne. - Our software is self-hosted by our customers and downstream providers. - We provide our customers extra support to ensure the software is secure, such as long-term support and early security notifications. - We cannot guarantee the security of the hardware of our customers and downstream providers. We cannot guarantee the security of the hardware as we only deliver software. Hardware security is the responsibility of those who host the software. - For internal use of Nextcloud GmbH, we use a well-respected EU-based hosting provider for our hardware (Hetzner) and a well-respected EU-based AI service provider (Ionos) with whom we have a data processing agreement. ## Additional requirements when using large AI models Nextcloud's AI products are designed to be used with smaller AI models that can also run on-premise. Nextcloud's AI Act compliance efforts thus assume you are using models that were trained using less than 10^25 floating point operations. However, Nextcloud's AI products are designed (in accordance with the AI act) to be interoperable and therefore it is technically possible to use larger models. If you decide to use larger models, this qualifies as a significant modification of the system and additional legal requirements for systemic-risk General purpose AI systems apply. Please refer to a lawyer and to [the EU AI Act](https://artificialintelligenceact.eu/gpai-guidelines-overview/comes).