Python Markdown using MkDocs: Kodi

Important

This WILL NOT give you access to films that you do not currently have access to. Kodi is being used to download information about the films from The Movie Database and then for this information to be extracted through the Kodi API. At no point will the films themselves be downloaded!

A simple example that uses t-python-markdown and MkDocs to create a simple static website using metadata about films extracted from Kodi.

This example uses VirtualBox to both host Kodi and create/run the example code. If you want to run the example code outside, then you may need to adjust the networking for the VM to allow external access.

Excluding download times for VirtualBox and Ubuntu, this should take roughly 30-45 minutes to complete.

This is not meant to be a tutorial for MkDocs, Kodi, VirtualBox or Ubuntu, but should give you a basic setup to work from.

Pre-Requisites

This example has been written and tested on Linux.

You will need Kodi, but if you don't already have it available, that will be covered below.

Also, if you haven't used Kodi before, then please read Kodi Basic Controls as that will help navigating Kodi to add and remove media.

Install and Configure Kodi

If you already have access to Kodi and are happy to use it for this then you can skip this step.

This example requires a running copy of Kodi. The following instructions should get you up and running with Kodi:

Download/Install VirtualBoxInstall Ubuntu DesktopInstall virtualenv on UbuntuInstall KodiConfigure Kodi

Go to VirtualBox and follow the appropriate instructions for your OS and install VirtualBox.

Go to this Ubuntu Tutorial and follow the instructions to install Ubuntu Desktop in VirtualBox.

If possible, try to allocate a minimum of 2 cores, 4GB RAM and 50GB hard disk.

• Start the Ubuntu VM and login
• Start the terminal and su - to login as root (the password will be the same as the account you created during "Install Ubunti Desktop")
• At the prompt, now type the following and press «enter»

  apt install virtualenv -y
  

• Once installed, close the terminal

• Start the Ubuntu VM and login
• Open "Ubuntu Software", search for "Kodi" and install it
• Once installed, close "Ubuntu Software"

The following steps will take you through adding the media folder, enabling clean up (useful when testing and when you remove media) and also the web interface (required to enable access the JSON-RPC API).

• Start the Ubuntu VM and login
• Start the File Manager and ensure that folder ~/Videos/films exists
• Start Kodi
• Add ~/Videos/films folder to Kodi:
  1. Go to the "Movies" section and select "Enter files section"
  2. Select "Files" and "Add videos..."
  3. Click "Browse" then "Home folder" / "Videos" / "films" and click "OK"
  4. Click "OK"
  5. Then on "Set content" change "This directory contains" to "Movies"
  6. Click "OK" to add it
  7. On "Change content" click "No" as at present there is no content
  8. Finally, press ESC a couple of times to return to the Kodi home screen
• Enable clean up:
  1. Find and click on the "Settings" icon () at the top of the left menu
  2. Now click on "Media"
  3. In the bottom left corner is a settings icon with "Standard" next to it. Click on this until it changes to "Advanced"
  4. Finally, press ESC a couple of times to return to the Kodi home screen
• Enable web interface:
  1. Find and click on the "Settings" icon () at the top of the left menu
  2. Now click on "Services" and then "Control"
  3. Click on "Password" and set this, for now, set it to kodi
  4. Enable "Allow remote control via HTTP". You will get a "Warning!" message about access to the web interface and that this device should never be exposted on the Internet. Good advice which should be followed. Click "Yes" to continue
  5. Finally, press ESC a couple of times to return to the Kodi home screen

MkDocs Installation

Use the following to initialise and configure a basic MkDocs setup:

• Start the Ubuntu VM and login
• Start the terminal
• Change to a directory where the folder python-markdown-mkdocs-kodi will be created

• Copy the following, paste it into the terminal and then press «enter»:

  virtualenv python-markdown-mkdocs-kodi
  cd python-markdown-mkdocs-kodi
  . ./bin/activate
  pip install t-python-markdown mkdocs mkdocs-material
  mkdir -p src/docs/film src/docs/people
  

Now create the following files:

mkdocs.ymltags.mdextra.css

Copy and save the following to python-markdown-mkdocs-kodi/src/mkdocs.yml:

site_name: !ENV [SITE_NAME, "My Kodi Media"]
site_author: t-python-markdown
use_directory_urls: false
theme:
  name: material
markdown_extensions:
  - pymdownx.emoji:
      emoji_index: !!python/name:materialx.emoji.twemoji
      emoji_generator: !!python/name:materialx.emoji.to_svg
  - md_in_html
  - attr_list
plugins:
  - search
  - tags:
      tags_file: tags.md
extra_css:
  - extra.css
nav:
  - Home: "index.md"

Copy and save the following to python-markdown-mkdocs-kodi/src/docs/tags.md:

---
title: Tags
search:
  exclude: true
hide:
- navigation
---

# Tags

[TAGS]

Copy and save the following to python-markdown-mkdocs-kodi/src/docs/extra.css:

hr {
  margin-top: 8px !important;
  margin-bottom: 8px !important;
}

a:hover {
  text-decoration: underline;
  text-decoration-color: #4e4e4e;
}

.plot {
  font-size: 1.2em;
}

.year-rating-runtime {
  font-size: 0.9em;
}

table {
  table-layout: fixed;
  display: table !important;
  border: 0px !important;
}

.genre-list>ul {
  list-style: none !important;
  margin-left: 0px !important;
  line-height: 40px;
}

.genre-list>ul>li {
  text-align: center;
  float: left;
  white-space: nowrap;
  text-decoration: none;
}

.genre-list>ul>li>a {
  color: inherit;
  text-decoration: none;
}

.cast-table>div>div>table>tbody>tr>td:nth-child(1) {
  height: 2.5em;
}

.cast-table>div>div>table>tbody>tr>td:nth-child(n+1) {
  vertical-align: middle;
  font-size: 1.2em;
}

.people-table>div>div>table>thead>tr>th:nth-child(1) {
  width: 130px;
}

.people-table>div>div>table>tbody>tr>td:nth-child(1) {
  height: 2.5em;
}

.people-table>div>div>table>tbody>tr>td:nth-child(n+1) {
  vertical-align: middle;
  font-size: 1.2em;
}

th {
  padding-top: 8px !important;
  padding-bottom: 8px !important;
  border-left: 0px !important;
  border-right: 0px !important;
  background-color: #f0f0f0 !important
}

td {
  padding-top: 2px !important;
  padding-bottom: 2px !important;
  border-left: 0px !important;
  border-right: 0px !important;
  border-top: 1px solid #f0f0f0 !important;
  border-bottom: 1px solid #f0f0f0 !important;
}

Starting MkDocs

Change directory to python-markdown-mkdocs-kodi/src and start mkdocs:

mkdocs serve

Once done, open a browser to http://127.0.0.1:8000. It should show, correctly, a 404 - Not found message. This means that the basics are now in place.

What The Code Does

This example uses the Kodi JSON-RPC API to retrieve a list of films (movies) which it then iterates through creating a single page for each film. In addition, for every cast member, writer and director, a single page is created for each that lists all the films they are involved with.

It demonstrates how to use t-python-markdown to quickly and easily generate markdown documents.

Python Code

Create a new file python-markdown-mkdocs-kodi/src/generate_site.py, then copy and save the following to it:

generate_site.py

"""Example usage of t-python-markdown using Kodi"""
import argparse
from collections import Counter
import re
import requests
from t_python_markdown import Document, Header, Table, Paragraph, Sentence, HorizontalRule, Bold, UnorderedList, Link


class Kodi:
  """Read/process details from Kodi"""
  __ROLE_ICONS = {"actor": ":speaking_head:", "writer": ":material-typewriter:", "director": ":clapper:"}

  def __init__(self, kodi_url, kodi_user, kodi_password):
    self.__kodi_url = kodi_url[:-1] if kodi_url.endswith("/") else kodi_url
    self.__genre_counts = {}
    self.__people = {}
    self.__kodi_user = kodi_user
    self.__kodi_password = kodi_password

  def process(self):
    """Build Site"""
    self.__render_film_pages()
    self.__render_people_pages()
    self.__render_main_page()

  def __render_main_page(self):
    """Add main page (index.md)"""
    doc = Document({"title": "Home", "search": {"exclude": "true"}, "hide": ["navigation", "toc", "tags"]})
    doc >> Header("Home", 1)
    doc >> "\n<div class='genre-list' markdown>"
    doc >> UnorderedList([Sentence([Link(_, f"tags.md#{_.replace(' ', '-').lower()}"), "{: .md-tag }"], separator="\n", end="") for _ in sorted(list(self.__genre_counts.keys()))])
    doc >> "</div>"
    doc.write("docs/index.md")

  def __render_film_pages(self):
    """Add film pages"""
    films = self.__get_films()
    for film in films:
      print(f"Processing film: '{film['title']}'")
      self.__render_media_page(film, "film")

  def __render_people_pages(self):
    """Add people pages"""
    for k, v in self.__people.items():
      self.__render_people_page(v)

  def __render_media_page(self, media_details: dict, media_format: str):
    """Add media page"""
    self.__increment_genre_counts(media_details["genre"])

    media_filename = self.__filename(f"{media_details['title']}")
    media_details.update({"_filename": media_filename, "_format": media_format})

    # Create document
    doc = Document({"title": media_details["title"], "hide": ["navigation"], "tags": media_details.get("genre", [])})

    # Add title and year/rating/runtime
    doc >> Header(f'{self.__format_icon(media_format)} {media_details["title"]}', 1)
    doc >> self.__calc_year_rating_runtime(media_details)

    # Add plot
    plot = media_details["plot"] if media_details["plot"] else "No plot available."
    plot_lines = list(filter(lambda l: l, plot.split("\n")))
    for para in [Paragraph([f"{_}\n{{ .plot }}"]) for _ in plot_lines]:
      doc >> para

    # Initialise director(s)
    directors = media_details.get("director", [])
    doc_directors = Document()
    doc >> doc_directors

    # Initialise writer(s)
    writers = media_details.get("writer", [])
    doc_writers = Document()
    doc >> doc_writers

    # Add tagline
    if media_details.get("tagline"):
      doc >> HorizontalRule()
      doc >> Paragraph([Bold("Tagline"), media_details["tagline"], "\n{ data-search-exclude }"])

    # Add cast
    if media_details["cast"]:
      cast_list = Table(["Name", "Role"], [":--"])
      for cast_member in media_details["cast"]:
        cast_link = Link(cast_member["name"], f"../people/{self.__add_job_role(cast_member['name'], 'actor', media_details, cast_member['order'])}.md")
        cast_list >> [cast_link, cast_member["role"]]
    else:
      cast_list = Paragraph("No cast available.")
    doc >> Header("Cast { data-search-exclude }", 2)
    doc >> "<div class='cast-table' markdown>"
    doc >> cast_list
    doc >> "</div>"

    # Add details
    doc >> Header("Details { data-search-exclude }", 2)
    if "country" in media_details:
      self.__detail_with_list(doc, ("Country", "Countries"), media_details["country"])
    doc >> Paragraph([Bold("Premiered"), media_details["premiered"]])
    self.__detail_with_list(doc, ("Studio", "Studios"), media_details["studio"], include_hr=False)

    # Add directors
    if directors:
      directors = Counter(directors)
      director_names = sorted(directors.keys(), key=lambda l: directors[l], reverse=True)
      directors = [Link(f"{_}{'' if directors[_] == 1 else f' ({directors[_]})'}", f"../people/{self.__add_job_role(_, 'director', media_details)}.md") for _ in director_names]
      self.__detail_with_list(doc_directors, ("Director", "Directors"), directors)

    # Add writers
    if writers:
      writers = Counter(writers)
      writer_names = sorted(writers.keys(), key=lambda l: writers[l], reverse=True)
      writers = [Link(f"{_}{'' if writers[_] == 1 else f' ({writers[_]})'}", f"../people/{self.__add_job_role(_, 'writer', media_details)}.md") for _ in writer_names]
      self.__detail_with_list(doc_writers, ("Writer", "Writers"), writers)

    doc.write(f"docs/{media_format}/{media_filename}.md")

  def __calc_year_rating_runtime(self, media_details):
    """Calculate year/rating/runtime for media"""
    rv = [str(media_details["year"])]
    if media_details.get("mpaa"):
      rv.extend(["-", media_details["mpaa"]])
    if media_details.get("runtime"):
      rv.extend(["-", self.__hm_time(media_details["runtime"])])
    if media_details.get("rating"):
      rv.extend(["- :star:", f"{media_details['rating']:.1f}"])
    rv.extend(["\n{ .year-rating-runtime }"])
    return Paragraph(rv)

  def __render_people_page(self, people):
    """Add people page"""
    doc = Document({"title": people["cast"], "hide": ["navigation"]})
    doc >> Header(f'{people["cast"].title()}', 1)
    for k, v in sorted(people["roles"].items()):
      doc >> Header(f"{Kodi.__ROLE_ICONS.get(k, ':grey_question:')} {k.title()}", 2)
      is_actor = k in ["actor"]
      table = Table(["Premiered", "TV/Film", "Role"] if is_actor else ["Premiered", "TV/Film"], ":--")
      doc >> "<div class='people-table' markdown>"
      doc >> table
      doc >> "</div>"
      for pr in sorted(v, key=lambda l: l["media_details"]["premiered"]):
        media_details = pr["media_details"]
        index = pr["index"]
        role = {"role": "-"} if index is None else next(iter(filter(lambda l: l["order"] == index, media_details.get("cast", []))))
        media_filename = media_details.get("_filename")
        media_format = media_details.get("_format")
        title = Sentence([self.__format_icon(media_details["_format"]), media_details.get("title")], end="")
        if media_details.get("rating"):
          title >> f":star: {media_details['rating']:.1f}"
        table_row = [media_details.get("premiered", "-"), Sentence(Link(title, f"../{media_format}/{media_filename}.md"), end="")]
        if is_actor:
          table_row.append(role["role"])
        table >> table_row
    doc.write(f"docs/people/{self.__filename(people['id'])}.md")

  def __format_icon(self, media_format):
    """Return appropriate emoji for media format"""
    return ":tv:" if media_format == "tv" else ":movie_camera:"

  def __filename(self, s: str):
    """Derive filename"""
    return re.sub('[^0-9a-zA-Z]+', '_', s).lower()

  def __increment_genre_counts(self, genres: list):
    """Increment genre counts"""
    for genre in genres:
      if genre not in self.__genre_counts:
        self.__genre_counts[genre] = 0
      self.__genre_counts[genre] += 1

  def __add_job_role(self, cast, role, media_details, index=None):
    cast_lower = cast.lower()
    cast_id = self.__filename(cast)
    if cast_lower not in self.__people:
      self.__people[cast_lower] = {"cast": cast, "id": cast_id, "roles": {}}
    cast_lower = self.__people[cast_lower]
    role_lower = role.lower()
    roles = cast_lower["roles"]
    if role_lower not in roles:
      roles[role_lower] = []

    roles[role_lower].append({"media_details": media_details, "index": index})
    return cast_id

  def __detail_with_list(self, doc, title, entries, include_hr=True):
    """Convenience method. Add detail with list of values"""
    if entries:
      if include_hr:
        doc >> HorizontalRule()
      doc >> Paragraph([Bold(f"{title[0 if len(entries)==1 else 1]}"), ", ".join([str(_) for _ in entries]), "\n{ data-search-exclude }"])

  def __hm_time(self, seconds: int) -> str:
    """Convert seconds to human readable hours/minutes"""
    m, s = divmod(seconds, 60)
    h, m = divmod(m, 60)
    return f"{h}h {m}m"

  def __get_kodi(self, params: dict):
    """Retrieve something from Kodi"""
    headers = {"content-type": "application/json"}
    resp = requests.post(
        f"{self.__kodi_url}/jsonrpc",
        json=params,
        headers=headers,
        auth=(self.__kodi_user, self.__kodi_password),
        timeout=30
    )
    return resp.json()

  def __get_films(self):
    """Retrieve list of films from Kodi"""
    params = {"jsonrpc": "2.0", "method": "VideoLibrary.GetMovies", "id": 0,
              "params": {
                  "properties": [
                      "title", "genre", "year", "rating", "director",
                      "trailer", "tagline", "plot", "plotoutline", "originaltitle",
                      "writer", "studio", "mpaa", "cast", "country",
                      "runtime", "set", "showlink", "top250", "votes",
                      "sorttitle", "setid", "tag", "art", "userrating",
                      "ratings", "premiered", "uniqueid",
                  ]}}
    return self.__get_kodi(params)["result"]["movies"]


if __name__ == "__main__":
  parser = argparse.ArgumentParser()
  parser.add_argument("--kodi-url", "-k", type=str, default="http://localhost:8080", help="Kodi URL")
  parser.add_argument("--user", "-u",     type=str, default="kodi",                  help="Kodi user")
  parser.add_argument("--password", "-p", type=str, default="kodi",                  help="Kodi Password")
  args = parser.parse_args()

  try:
    Kodi(kodi_url=args.kodi_url, kodi_user=args.user, kodi_password=args.password).process()
  except Exception as e:
    print(f"Problem occurred: {e}")
    raise

Add Films to Kodi

Important

Just a reminder. This WILL NOT give you access to films that you do not currently have access to. Kodi is being used to download information about the films from The Movie Database and then for this information to be extracted through the Kodi API. At no point will the films themselves be downloaded!

To get Kodi to retrieve the details of a film, all that is required is for an appropriately named file (name + year) to be created in ~/Videos/films/. For example, Das Boot (1981).mkv, Seven Samurai (1954).mkv, etc.

The following will simplify that by processing the contents of a file where each line is the name of a film. For example, if you like Lord of the Rings, then create a file called ~/Videos/films/lotr.txt with the following:

The Lord of the Rings The Fellowship of the Ring (2001)
The Lord of the Rings The Return of the King (2003)
The Lord of the Rings The Two Towers (2002)

Then at the terminal, change directory to ~/Videos/films/ and paste the following changing {{FILE}} for lotr.txt.

while IFS= read -r line; do touch "$line.mkv"; done < {{FILE}}

Once done, go to Kodi and run an update.

Note that the more films you add, the longer the site could take to start. Start small and add more once you have things up and running

A few film lists to get you started...

ActorsCollections

• Angelina Jolie
• Arnold Schwarzenegger
• Dwayne Johnson
• Emily Blunt
• Helen Mirren
• Humphrey Bogart
• Jackie Chan
• Kevin Bacon
• Michael Caine

• IMDB Top 250
• IMDB Top 1000 Films 20th Century
• IMDB Top 1000 Films 21st Century
• Batman
• DC
• James Bond
• JRR Tolkien
• Marvel
• Philip K Dick
• Star Trek
• Star Wars

How to update Kodi

When you add any films, you will need to update Kodi. When you remove films, you will need to clean the library to remove those that you removed. There are multiple ways of achieving both these activities, the following should help you get started if you haven't used Kodi before:

• Add or Remove Films Select "Movies" from the Kodi home screen which should take you to "Movie / Titles". Now press the left arrow on the keyboard or click on "Options". From the menu that pops out, under "Actions" select "Update library". Wait until the update completes before proceeding
• Remove Films From the Kodi home screen, click on the "Settings" icon () at the top of the left menu. Click on "Media", then in "Library" you should see "Clean library", select this. When prompted, select "Yes" to clean the library and wait. Also, remove any generated files in src/docs/film and src/docs/people before running the python code (see below)

Extract The Films and Build the Website

Start the terminal, change directory to python-markdown-mkdocs-kodi and ensure that you are running in a virtual environment as follows:

. ./bin/activate

Then change directory to src and run the following:

python generate_site.py

If you don't run this on the Kodi VM or you are using your own Kodi, then you will need to use the --kodi-url argument to pass the IP address and port of your Kodi instance. Also, if you used different credentials for Kodi (kodi/kodi if you followed the instructions above), then you will need to use the --user and --password arguments to pass the correct credentials.

Afterwards, the website should change to show a list of the films added to Kodi.

If you don't currently have mkdocs running, see Starting MkDocs above

Further Exercises

Improve the Layout

Change the layout of the site. This example has a very basic layout. See what you can do to improve it.

Add Support for TV

Extend the generator to include TV shows from Kodi. I have included the Kodi API calls below to get you started. But you will need to add the relevant calls to these and integrate the responses in the page rendering.

Retrieve TV shows and Episodes from Kodi

def __kodi_get_tv_shows(self):
  """Retrieve list of TV shows from Kodi"""
  params = {"jsonrpc": "2.0", "method": "VideoLibrary.GetTVShows", "id": 0,
            "params": {
                "properties": [
                    "title", "genre", "year", "rating", "plot",
                    "studio", "mpaa", "cast", "playcount", "episode",
                    "imdbnumber", "premiered", "votes", "lastplayed", "fanart",
                    "thumbnail", "originaltitle", "sorttitle", "episodeguide", "season",
                    "watchedepisodes", "dateadded", "tag", "art", "userrating",
                    "ratings", "runtime", "uniqueid"
                ]}}
  return self.__get_kodi(params)["result"]["tvshows"]

def __get_tv_episodes(self, show_id: int):
  """Retrieve list of TV episodes from Kodi"""
  params = {"jsonrpc": "2.0", "method": "VideoLibrary.GetEpisodes", "id": 0,
            "params": {
                "tvshowid": show_id,
                "properties": [
                    "title", "plot", "votes", "rating", "writer",
                    "firstaired", "playcount", "runtime", "director", "productioncode",
                    "season", "episode", "originaltitle", "showtitle", "cast",
                    "streamdetails", "lastplayed", "fanart", "thumbnail", "file",
                    "resume", "tvshowid", "dateadded", "uniqueid", "art",
                    "specialsortseason", "specialsortepisode", "userrating", "seasonid", "ratings"
                ]}}
  return self.__get_kodi(params)["result"].get("episodes", [])

Build a Website

Use MkDocs to build a website instead:

mkdocs build

and then take the generated output and deploy that using a web server of your choice.

GitLab Pages

If like me, you use GitLab, add this code to a GitLab project and create a Gitlab pipeline to deploy this via GitLab Pages.

Be careful, dependent on how many films you add to Kodi, the result could be sizeable and not upload correctly without adjusting the Pages configuration in Gitlab

Download Images from Kodi

Download the poster or fanart images from Kodi and include them on the media page.

Downloading images takes up a lot of disk space. If you do all of this in the VM, then these images will be stored 2 or 3 times. Once in Kodi, once in the docs folder and if you build the site, again in the site folder. Just make sure you have sufficient space available before including them.