{
"cells": [
{
"cell_type": "markdown",
"id": "25373f1b-f4bf-4cee-8c45-952d9a6118a7",
"metadata": {},
"source": [
"# Table detection with `OPENCV_CONTOURS`\n",
"\n",
":::{warning}\n",
"This is an *experimental*, alternate method for table detection. The method is promising, but it has some rough edges which\n",
"needs to be fixed before it can be used.\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "9c1d358f-a58a-4298-9d88-22c6c6ce5f2b",
"metadata": {},
"source": [
"The algorithm behind this method can be approximately described as follows:\n",
"\n",
"- We start from a grayscale pre-processed image with textremoved and tables identified using {func}`scipy.ndimage.label` as in [](./table_detect_scipy_proj.myst.ipynb).\n",
"- The function {func}`dawsonia.table_detect.opencv_contours.get_table_structure` calls `cv2.findContours` to identify contours. The bounding boxes (bbox) are saved instead of the exact curvilinear contour. Unusual bboxes (too small, too big, strange aspect ratios) are weeded out,\n",
"- Clustering algorithm {func}`dawsonia.table_detect.opencv_contours.cluster_axis` identifies columns and rows\n",
"- Column clusters and row clusters are intersected to form a 3D array, with shape `(nb_rows, nb_columns, 4)` in {func}`dawsonia.table_detect.opencv_contours.create_bbox_array`.\n",
"- Once again unusual bboxes (width or height deviates too much from the median in the column / row) are removed using {func}`dawsonia.table_detect.opencv_contours.set_nan_to_odd_bboxes`.\n",
"- Missing bboxes are discovered using a cubic curve fit in {func}`dawsonia.table_detect.opencv_contours.fix_missing_bboxes`. \n",
"\n",
"To activate this the table configuration file need to be altered. But first, we start by setting our environment"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "3f1f14f1-121a-48b4-a08a-6cb5b8a97859",
"metadata": {},
"outputs": [],
"source": [
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "89c199bb-77cc-47dd-8951-776b0bc03e83",
"metadata": {},
"outputs": [],
"source": [
"%load_ext autoreload"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "caf43db9-aea6-4de3-90ac-dd14eff3ab64",
"metadata": {},
"outputs": [],
"source": [
"autoreload 2"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "19122e68-f286-40cc-891a-23beba633007",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"os.environ[\"DAWSONIA_DEBUG_TABLE_DETECT\"] = \"1\""
]
},
{
"cell_type": "markdown",
"id": "1367c007-4af0-47db-afe8-d239ac788d70",
"metadata": {},
"source": [
"Now we write a new _table formats_ file. Notice the new key `method` under `[default.preproc]`."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "4f33e0ef-6173-437b-926f-da2cd59ff57c",
"metadata": {},
"outputs": [],
"source": [
"mkdir -p table_formats"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "4b347a6a",
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing table_formats/bjuröklubb_example.toml\n"
]
}
],
"source": [
"%%file table_formats/bjuröklubb_example.toml\n",
"[default]\n",
"version = 0\n",
"\n",
"# Default values, but wrote explicitly here. See PreprocConfig class\n",
"[default.preproc]\n",
"table_modif = true\n",
"corr_rotate = true\n",
"## NOTE: Here we alter the method\n",
"method = \"OPENCV_CONTOURS\"\n",
"row_idx_unit = \"HOURS\"\n",
"idx_tables_size_verify = [0, 1]\n",
"\n",
"[1927]\n",
"version = 1\n",
"\n",
"[version.0]\n",
"columns = [\n",
" [\n",
" \"term_på_baro\",\n",
" \"barom\",\n",
" \"torra_term\",\n",
" \"våta_term\",\n",
" \"moln_slag_lägre\",\n",
" \"moln_mängd_lägre\",\n",
" \"moln_slag_medel\",\n",
" \"moln_slag_högre\"\n",
" ],\n",
" [\n",
" \"moln_het_sol_dimma_nederbörd_total\",\n",
" \"vind_riktning\",\n",
" \"vind_beaufort\",\n",
" \"vind_m_sek\",\n",
" \"sikt\",\n",
" \"sjögang\",\n",
" \"maximi_term\",\n",
" \"minimi_term\",\n",
" \"nederbörd_mängd\",\n",
" \"nederbörd_slag\"\n",
" ]\n",
"]\n",
"name_idx = \"tid\"\n",
"rows = [2, 8, 14, 19, 21]\n",
"tables = [\n",
" [5, 8],\n",
" [5, 10],\n",
" [3, 1],\n",
" [4, 2],\n",
" [4, 5]\n",
"]\n",
"\n",
"[version.1]\n",
"columns = [\n",
" [\n",
" \"term_på_baro\",\n",
" \"barom\",\n",
" \"torra_term\",\n",
" \"våta_term\",\n",
" \"moln_slag_lägre\",\n",
" \"moln_mängd_lägre\",\n",
" \"moln_slag_högre\",\n",
" \"moln_mängd_total\"\n",
" ],\n",
" [\n",
" \"vind_riktning\",\n",
" \"vind_beaufort\",\n",
" \"vind_m_sek\",\n",
" \"sikt\",\n",
" \"sjögang\",\n",
" \"maximi_term\",\n",
" \"minimi_term\",\n",
" \"nederbörd_mängd\",\n",
" \"nederbörd_slag\"\n",
" ]\n",
"]\n",
"rows = [2, 8, 14, 19, 21]\n",
"tables = [\n",
" [5, 8],\n",
" [5, 9],\n",
" [3, 1],\n",
" [4, 2],\n",
" [4, 5]\n",
"]"
]
},
{
"cell_type": "markdown",
"id": "802c06bd-68a2-43a6-b6d1-16b3264c9de5",
"metadata": {},
"source": [
"## Demo: Table detection\n",
"\n",
"Now that we have all the prerequisites read, let's execute the command:\n",
"\n",
"```sh\n",
"DAWSONIA_DEBUG_TABLE_DETECT=1 dawsonia label --first-page 4 --last-page 4 --no-interactive data/raw_zarr/bjuröklubb_example/bjuröklubb_1927.zarr\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "2faa23c0-4b5c-4dc5-a5f0-d97ad9fab5c2",
"metadata": {},
"source": [
"or its Python API equivalent:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "a52bd0bc-47ab-4992-b9c6-76ab04243210",
"metadata": {
"editable": true,
"slideshow": {
"slide_type": ""
},
"tags": []
},
"outputs": [
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:14 ,892 - dawsonia.io._zarr - INFO - table_format = TableFormat ( name_idx ='tid' , \n",
" columns =[[ 'term_på_baro' , 'barom' , 'torra_term' , 'våta_term' , 'moln_slag_lägre' , 'moln_mängd_lägre' , \n",
" 'moln_slag_högre' , 'moln_mängd_total' ] , [ 'vind_riktning' , 'vind_beaufort' , 'vind_m_sek' , 'sikt' , \n",
" 'sjögang' , 'maximi_term' , 'minimi_term' , 'nederbörd_mängd' , 'nederbörd_slag' ]] , rows =( datetime.time ( 2 , 0 ) ,\n",
" datetime.time ( 8 , 0 ) , datetime.time ( 14 , 0 ) , datetime.time ( 19 , 0 ) , datetime.time ( 21 , 0 )) , tables =[[ 5 , 8 ] , \n",
" [ 5 , 9 ] , [ 3 , 1 ] , [ 4 , 2 ] , [ 4 , 5 ]] , preproc =PreprocConfig ( table_modif =True , corr_rotate =True , \n",
" row_idx_unit =< TimeUnits.HOURS: 1 >, method =<PreprocMethods.OPENCV_CONTOURS: 2 > , idx_tables_size_verify =[ 0 , \n",
" 1 ]) , transforms =None , version ='1' , station ='bjuröklubb_example' ) \n",
"\u001b[0m, \u001b[33midx_tables_size_verify\u001b[0m=\u001b[1m[\u001b[0m\u001b[1;36m0\u001b[0m, \n",
" \u001b[1;36m1\u001b[0m\u001b[1m]\u001b[0m\u001b[1m)\u001b[0m, \u001b[33mtransforms\u001b[0m=\u001b[3;35mNone\u001b[0m, \u001b[33mversion\u001b[0m=\u001b[32m'1'\u001b[0m, \u001b[33mstation\u001b[0m=\u001b[32m'bjuröklubb_example'\u001b[0m\u001b[1m)\u001b[0m \n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:14 ,913 - dawsonia.io._zarr - INFO - Setting first_page = 4 \n",
"INFO 2024 -06 -02 22:02:14 ,919 - dawsonia.io._zarr - INFO - Setting first_page = 4 \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:21 ,643 - dawsonia.table_detect.opencv_contours - INFO - Finding contours for a mask with \n",
" table_nb_pixels =19366 \n",
"INFO 2024 -06 -02 22:02:21 ,655 - dawsonia.table_detect.opencv_contours - INFO - No bboxes detected! \n",
"INFO 2024 -06 -02 22:02:21 ,678 - dawsonia.table_detect.opencv_contours - INFO - Finding contours for a mask with \n",
" table_nb_pixels =75059 \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:23 ,325 - dawsonia.table_detect.opencv_contours - INFO - cluster labels = [ 3 1 4 5 9 \n",
" 6 10 2 0 7 8 ] \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:24 ,159 - dawsonia.table_detect.opencv_contours - INFO - cluster labels = [ 6 0 1 5 7 3 2 \n",
" 4 ] \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/a002487/.pyenv/versions/3.9.13/envs/ai-for-obs/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1217: RuntimeWarning: All-NaN slice encountered\n",
" return function_base._ureduce(a, func=_nanmedian, keepdims=keepdims,\n"
]
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:25 ,039 - dawsonia.table_detect.opencv_contours - INFO - Dumped ( 'bboxes' , \n",
" 'column_labels' , 'column_bboxes_idxs' , 'row_labels' , 'row_bboxes_idxs' , 'original_image' ) into \n",
" /home/a002487/Sources/ai-for-obs/dawsonia/docs/source/getting_started/ get_table_structure_07ca022c-211b-11 \n",
" ef-98fa-84c5a6cd0398.joblib. Explore it using get_table_strucutre_interpolation.ipynb \n",
"INFO 2024 -06 -02 22:02:25 ,055 - dawsonia.table_detect.opencv_contours - INFO - Finding contours for a mask with \n",
" table_nb_pixels =70550 \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:26 ,971 - dawsonia.table_detect.opencv_contours - INFO - cluster labels = [ 8 0 10 5 1 \n",
" 2 4 6 3 9 7 ] \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:27 ,894 - dawsonia.table_detect.opencv_contours - INFO - cluster labels = [ 6 0 5 8 3 4 7 2 \n",
" 1 ] \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/a002487/.pyenv/versions/3.9.13/envs/ai-for-obs/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1217: RuntimeWarning: All-NaN slice encountered\n",
" return function_base._ureduce(a, func=_nanmedian, keepdims=keepdims,\n"
]
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:28 ,921 - dawsonia.table_detect.opencv_contours - INFO - Dumped ( 'bboxes' , \n",
" 'column_labels' , 'column_bboxes_idxs' , 'row_labels' , 'row_bboxes_idxs' , 'original_image' ) into \n",
" /home/a002487/Sources/ai-for-obs/dawsonia/docs/source/getting_started/ get_table_structure_0a1a75fc-211b-11 \n",
" ef-98fa-84c5a6cd0398.joblib. Explore it using get_table_strucutre_interpolation.ipynb \n",
"INFO 2024 -06 -02 22:02:28 ,955 - dawsonia.table_detect.opencv_contours - INFO - Finding contours for a mask with \n",
" table_nb_pixels =46857 \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:30 ,577 - dawsonia.table_detect.opencv_contours - INFO - cluster labels = [ 2 3 1 0 ] \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:31 ,400 - dawsonia.table_detect.opencv_contours - INFO - cluster labels = [ 4 0 2 1 3 ] \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"ERROR 2024 -06 -02 22:02:32 ,228 - dawsonia.table_detect.opencv_contours - ERROR - Error occured while attempting \n",
" to curve fit with 3 data points. Defaulting to median value... \n",
"ERROR 2024 -06 -02 22:02:32 ,235 - dawsonia.table_detect.opencv_contours - ERROR - TypeError ( 'The number of func \n",
" parameters=4 must not exceed the number of data points=3' ) \n",
"INFO 2024 -06 -02 22:02:32 ,249 - dawsonia.table_detect.opencv_contours - INFO - Dumped ( 'bboxes' , \n",
" 'column_labels' , 'column_bboxes_idxs' , 'row_labels' , 'row_bboxes_idxs' , 'original_image' ) into \n",
" /home/a002487/Sources/ai-for-obs/dawsonia/docs/source/getting_started/ get_table_structure_0c168e22-211b-11 \n",
" ef-98fa-84c5a6cd0398.joblib. Explore it using get_table_strucutre_interpolation.ipynb \n",
"INFO 2024 -06 -02 22:02:32 ,272 - dawsonia.table_detect.opencv_contours - INFO - Finding contours for a mask with \n",
" table_nb_pixels =49994 \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:33 ,575 - dawsonia.table_detect.opencv_contours - INFO - cluster labels = [ 2 0 1 3 4 ] \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:34 ,751 - dawsonia.table_detect.opencv_contours - INFO - cluster labels = [ 1 3 2 0 ] \n",
""
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"INFO 2024 -06 -02 22:02:35 ,810 - dawsonia.table_detect.opencv_contours - INFO - Dumped ( 'bboxes' , \n",
" 'column_labels' , 'column_bboxes_idxs' , 'row_labels' , 'row_bboxes_idxs' , 'original_image' ) into \n",
" /home/a002487/Sources/ai-for-obs/dawsonia/docs/source/getting_started/ get_table_structure_0e35f5c6-211b-11 \n",
" ef-98fa-84c5a6cd0398.joblib. Explore it using get_table_strucutre_interpolation.ipynb \n",
"INFO 2024 -06 -02 22:02:35 ,895 - dawsonia.table_detect.opencv_contours - INFO - Finding contours for a mask with \n",
" table_nb_pixels =21511 \n",
"INFO 2024 -06 -02 22:02:35 ,909 - dawsonia.table_detect.opencv_contours - INFO - No bboxes detected! \n",
"INFO 2024 -06 -02 22:02:35 ,998 - dawsonia.table_detect.opencv_contours - INFO - Preproc: applying \n",
" table_modif_rm_row_col \n",
"INFO 2024 -06 -02 22:02:36 ,003 - dawsonia.table_detect.utils - INFO - 🌧 no tables found \n",
"INFO 2024 -06 -02 22:02:36 ,015 - dawsonia.label - INFO - Table 0 not detected \n",
"INFO 2024 -06 -02 22:02:36 ,021 - dawsonia.label - INFO - Table 1 not detected \n",
"INFO 2024 -06 -02 22:02:36 ,027 - dawsonia.label - INFO - Table 2 not detected \n",
"INFO 2024 -06 -02 22:02:36 ,033 - dawsonia.label - INFO - Table 3 not detected \n",
"INFO 2024 -06 -02 22:02:36 ,039 - dawsonia.label - INFO - Table 4 not detected \n",
"