Backend fails to start with ModuleNotFoundError: No module named 'surfsense_backend' using Docker Compose

[rmc8]

Description:
When installing and starting SurfSense using Docker Compose, the backend service fails to start due to a ModuleNotFoundError and the container exits. This prevents the frontend from accessing the backend API (e.g., initiating Google authentication), rendering the application unusable.

This issue likely contributes to the root cause of the Workspace failure during Google authentication reported in existing Issue #48 (Google oauth Authorization URL).

Environment:

• OS: Ubuntu25.04
• Installation Method: Docker Compose

Steps to reproduce:

1. Clone the SurfSense repository.
2. Ensure Docker and Docker Compose are installed.
3. Complete the prerequisites according to the documentation (PGVector, Google OAuth setup, obtain API keys, etc.).
4. Configure the backend's .env file (surfsense_backend/.env) with the necessary environment variables (e.g., DATABASE_URL="postgresql+asyncpg://postgres:postgres@db:5432/surfsense", various API keys).
5. (If necessary) Adjust the backend port mapping in docker-compose.yml (e.g., ports: - "8800:8000").
6. Run docker compose up --build from the repository root directory in the foreground.

Expected behavior:
The backend container starts successfully without errors and remains in a running state. Logs indicating Uvicorn is listening on the designated port should appear.

Actual behavior:
The backend container fails to start, outputs the following ModuleNotFoundError, and exits.

backend-1   | 2025-05-08 04:19:29 - httpx - INFO - HTTP Request: GET https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json "HTTP/1.1 200 OK"
backend-1   | /usr/local/lib/python3.12/site-packages/chonkie/embeddings/auto.py:87: UserWarning: Error during registry lookup/instantiation: No matching embeddings implementation found for mixedbread-ai/mxbai-embed-large-v1. Falling back to SentenceTransformerEmbeddings.
backend-1   |   warnings.warn(
backend-1   | /usr/local/lib/python3.12/site-packages/chonkie/embeddings/auto.py:97: UserWarning: Falling back to SentenceTransformerEmbeddings.
backend-1   |   warnings.warn("Falling back to SentenceTransformerEmbeddings.")
backend-1   | 2025-05-08 04:19:35 - sentence_transformers.SentenceTransformer - INFO - Use pytorch device_name: cpu
backend-1   | 2025-05-08 04:19:35 - sentence_transformers.SentenceTransformer - INFO - Load pretrained SentenceTransformer: mixedbread-ai/mxbai-embed-large-v1
backend-1   | 2025-05-08 04:19:48 - sentence_transformers.SentenceTransformer - INFO - 2 prompts are loaded, with the keys: ['query', 'passage']
backend-1   | /usr/local/lib/python3.12/site-packages/chonkie/chunker/code.py:79: UserWarning: The language is set to `auto`. This would adversely affect the performance of the chunker. Consider setting the `language` parameter to a specific language to improve performance.
backend-1   |   warnings.warn("The language is set to `auto`. This would adversely affect the performance of the chunker. " +
backend-1   | Downloading https://github.com/zackees/ffmpeg_bins/raw/main/v5.0/linux.zip -> /usr/local/lib/python3.12/site-packages/static_ffmpeg/bin/linux.zip
backend-1   | 
backend-1   | Download of https://github.com/zackees/ffmpeg_bins/raw/main/v5.0/linux.zip -> /usr/local/lib/python3.12/site-packages/static_ffmpeg/bin/linux.zip completed.
backend-1   | Extracting /usr/local/lib/python3.12/site-packages/static_ffmpeg/bin/linux.zip -> /usr/local/lib/python3.12/site-packages/static_ffmpeg/bin
backend-1   | Loading FlashRankRanker model ms-marco-MiniLM-L-12-v2 (this message can be suppressed by setting verbose=0)
backend-1   | Loading model FlashRank model ms-marco-MiniLM-L-12-v2...
backend-1   | Traceback (most recent call last):
backend-1   |   File "/app/main.py", line 16, in <module>
backend-1   |     uvicorn.run(
backend-1   |   File "/usr/local/lib/python3.12/site-packages/uvicorn/main.py", line 580, in run
backend-1   |     server.run()
backend-1   |   File "/usr/local/lib/python3.12/site-packages/uvicorn/server.py", line 66, in run
backend-1   |     return asyncio.run(self.serve(sockets=sockets))
backend-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-1   |   File "/usr/local/lib/python3.12/asyncio/runners.py", line 195, in run
backend-1   |     return runner.run(main)
backend-1   |            ^^^^^^^^^^^^^^^^
backend-1   |   File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
backend-1   |     return self._loop.run_until_complete(task)
backend-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-1   |   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
backend-1   |   File "/usr/local/lib/python3.12/site-packages/uvicorn/server.py", line 70, in serve
backend-1   |     await self._serve(sockets)
backend-1   |   File "/usr/local/lib/python3.12/site-packages/uvicorn/server.py", line 77, in _serve
backend-1   |     config.load()
backend-1   |   File "/usr/local/lib/python3.12/site-packages/uvicorn/config.py", line 435, in load
backend-1   |     self.loaded_app = import_from_string(self.app)
backend-1   |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-1   |   File "/usr/local/lib/python3.12/site-packages/uvicorn/importer.py", line 22, in import_from_string
backend-1   |     raise exc from None
backend-1   |   File "/usr/local/lib/python3.12/site-packages/uvicorn/importer.py", line 19, in import_from_string
backend-1   |     module = importlib.import_module(module_str)
backend-1   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-1   |   File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
backend-1   |     return _bootstrap._gcd_import(name[level:], package, level)
backend-1   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-1   |   File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
backend-1   |   File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
backend-1   |   File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
backend-1   |   File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
backend-1   |   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
backend-1   |   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
backend-1   |   File "/app/app/app.py", line 18, in <module>
backend-1   |     from app.routes import router as crud_router
backend-1   |   File "/app/app/routes/__init__.py", line 4, in <module>
backend-1   |     from .podcasts_routes import router as podcasts_router
backend-1   |   File "/app/app/routes/podcasts_routes.py", line 10, in <module>
backend-1   |     from app.tasks.podcast_tasks import generate_chat_podcast
backend-1   |   File "/app/app/tasks/podcast_tasks.py", line 7, in <module>
backend-1   |     from surfsense_backend.app.agents.podcaster.state import State
backend-1   | ModuleNotFoundError: No module named 'surfsense_backend'
backend-1 exited with code 1

Specifically, the traceback indicates the error occurs in an import statement on line 7 of the /app/app/tasks/podcast_tasks.py file.

Root cause identification:
Based on the error log and the source code structure, the root cause appears to be an incorrect import path in the /surfsense_backend/app/tasks/podcast_tasks.py file. The import statement from surfsense_backend.app.agents.podcaster.state import State is likely incorrect relative to the container's file structure where /app is the root directory. Due to the docker-compose.yml volume mapping (volumes: - ./surfsense_backend:/app), the content of the host's ./surfsense_backend directory is mounted to the container's /app. Inside the container, there is no surfsense_backend directory directly under /app; instead, there is an app directory under /app. Therefore, the import path is incorrect.

Relevant code location:

SurfSense/surfsense_backend/app/tasks/podcast_tasks.py

Line 7 in 704af3e

from surfsense_backend.app.agents.podcaster.state import State

Solution/Workaround:
The issue can be resolved by modifying the import statement on line 7 of the /surfsense_backend/app/tasks/podcast_tasks.py file on the host machine as follows:

Original import statement:

from surfsense_backend.app.agents.podcaster.state import State

Corrected import statement:

from app.agents.podcaster.state import State

Applying this fix to the host file and then running docker compose up --build again allows the backend container to start successfully.

Related Issue:

• Google oauth Authorization URL #48 - Google oauth Authorization URL
  The Workspace failure during Google authentication reported in this Issue might be caused by the backend not being able to start. The ModuleNotFoundError identified here is likely the direct reason for the backend startup failure, and resolving this Issue may resolve the symptoms reported in Google oauth Authorization URL #48.

[MODSetter]

Thanks for reporting. This #76 should solve this.

[MODSetter]

@rmc8 LMK if its fixed.

[rmc8]

Closing as this is being addressed in #76.
Thanks for the quick turnaround!
Regarding #76, consider adding ruff checks to your CI workflow (e.g., GitHub Actions). This could catch import errors like this automatically before commits are pushed or merged.