Design a service on FastAPI: it receives a trigger on a schedule, processes PDF documents from Kafka (OCR, entity extraction), and saves the text + entities in Postgres. How would you design the structure?

For such a service on FastAPI, I would design the architecture as follows:

1. Scheduled Trigger

• Use APScheduler or Celery Beat to run periodic tasks that initiate processing.

2. Processing messages from Kafka

• Create a separate consumer that listens to the topic with PDF documents.
• Upon receiving a message, download or obtain the PDF (if the message contains a link or binary data).

3. OCR and entity extraction

• Use Tesseract or cloud services (Google Vision, AWS Textract) for OCR.
• For entity extraction, use NLP libraries (spaCy, DeepPavlov) or custom models.

4. Saving to Postgres

• Table documents with fields: id, filename, upload date, original text.
• Table entities with fields: id, document_id (FK), entity type, value, position in text.

5. Project structure

/app
  /api.py          # FastAPI endpoints
  /kafka_consumer.py # Message consumption logic
  /ocr.py          # OCR functions
  /ner.py          # Entity extraction
  /db.py           # Working with Postgres via SQLAlchemy
  /scheduler.py    # Task scheduler

6. Example Kafka consumer code (simplified):

from kafka import KafkaConsumer
import json

consumer = KafkaConsumer('pdf_topic', bootstrap_servers='localhost:9092')

for msg in consumer:
    data = json.loads(msg.value)
    pdf_path = data['pdf_path']
    text = ocr_process(pdf_path)
    entities = extract_entities(text)
    save_to_db(pdf_path, text, entities)

7. FastAPI can be used for monitoring processing status or manually triggering tasks.

Thus, the service will be modular, scalable, and easy to maintain.