Don't Break The Chain Peeps! Reblog Cause I'm Looking For Inspiration For My Next Masterpiece! 🙇🙇🙇

train on water

Story Map for Noobs: Cascade | WWF Network

Story Map is a web application template product that has been popularized in ArcGIS Online for a user-friendly and comprehensive narrative of maps. The ‘Cascade’ template has become the seamless interface of choice due to it’s ribbon transitions and availability of content streaming from external sources. 

Please refer to the following link for resources used in this webinar:

Story Map for Noobs: Cascade web application

📌 Availability: Retracted in 2021

Python: Geospatial Environment Setup (Part 2)

Python: Geospatial Environment Setup (Part 2)

Hey again folks! I am here for the second part of Python environmental setup for a geospatial workspace. I published the first part of this post two weeks ago. So if you've not yet read that, I'll catch you up to speed with our checklist:

Install Python ☑

Install Miniconda ☑

Install the basic Python libraries ☑

Create a new environment for your workspace

Install geospatial Python libraries

🗃 Create a new environment for your workspace

Since we have actually manually set up our base environment quite thoroughly with all the basic libraries needed, to make our work easier, we can just clone the base environment and install all the additional essential libraries needed for geospatial analysis. This new environment will be called geopy. Feel free to use a name you identify most with.

Why don't we just create a new environment? Well, it means we have to start installing the Python libraries again from scratch. Although it is no trouble to do so, we want to avoid installing so many libraries all at once. As I mentioned in Part 1, there is always a risk where incomplete dependencies in one library will affect the installation of other libraries that you intend to install in one go. Since we already have a stable and usable base environment, we can proceed to use it as a sort of pre-made skeleton that we will build our geospatial workspace with.

1️⃣ At the Anaconda Command Prompt, type the following:

2️⃣ Press Enter and the environment will be clone for you. Once it is done, you can use the following command to check the availability of your environment 👇🏻

You should be able to see your geopy environment listed along with the base environment.

👩🏻‍💻 Install geospatial Python libraries

Here we will proceed with the installation of a few geospatial Python libraries that are essential to reading and exploring the vectors and rasters.

🔺 fiona: This library is the core that some of the more updated libraries depend on. It is a simple and straightforward library that reads and writes spatial data in the common Python IOs without relying on the infamous GDAL's OGR classes.

🔺 shapely: shapely library features the capability to manipulate and edit spatial vector data in the planar geometric plane. It is one of the core libraries that recent geospatial Python libraries rely on to enable the reading and editing of vector data.

🔺 pyproj: is the Python interface for the cartographic projections and coordinate system libraries. Another main library that enables the 'location' characteristics in your spatial data to be read.

🔺 rasterio: reads and writes raster formats and provides a Python API based on Numpy N-dimensional arrays and GeoJSON.

🔺 geopandas: extends the pandas library to allow spatial operations on the geometric spatial data i.e shapefiles.

💀 As you might have noticed, we won't be doing any direct gdal library installation. It's mainly due to the fact that its installation is a process that seems to be accompanied by misery at every turn and involved workarounds that are pretty inconsistent for different individuals. Does it mean that we won't be using it for our Pythonic geospatial analysis? Heck no. But we will be taking advantage of the automatic dependency installation that comes with all the libraries above. The rasterio library depends on gdal and by installing it, we integrate the gdal library indirectly into our geospatial environment. I found that this method is the most fool-proof. Let's proceed to the installation of these libraries.

1️⃣ At the Anaconda Command Prompt, should you start from the beginning, ensure that your geopy environment is activated. If not, proceed to use the following command to activate geopy.

Once activated, we can install the libraries mentioned one after another. Nevertheless, you also have the option of installing them in one go directly using a single command 👇🏻

💀 geopandas is not included in this line-up NOT because we do not need it. It's another temperamental library that I prefer to isolate and install individually. If gdal is a rabid dog...then geopandas is a feral cat. You never know how-when-why it doesn't like you and forces a single 10-minute installation drag to hours.

3️⃣ Once you're done with installing the first line-up above, proceed with our feral cat below 👇🏻

4️⃣ Use the conda list command again to check if all the libraries have been installed successfully.

🎉Et voilá! Tahniah! You did it!🎉

🎯 The Jupyter Notebook

It should be the end of the road for the helluva task of creating the geospatial environment. But you're going to ask how to start using it anyway. To access this libraries and start analyzing, we can easily use the simple and straight-forward Jupyter Notebook. There are so many IDE choices out there but for data analysis, Jupyter Notebook suffices for me so far and if you are not familiar with Markdown, this tool will ease you into it slowly.

Jupyter Notebook can be installed in your geopy environment as follows:

And proceed to use it by prompting it open via the command prompt

It ain't that bad, right? If you're still having problems with the steps, do check out the real-time video I created to demonstrate the installation. And feel free to share with us what sort of problems you have encountered and the workaround or solutions you implemented! It's almost never a straight line with this, trust me. As mentioned in the previous post, check out the quick demo below 👇🏻

See you guys again for another session on geospatial Python soon!

wildlife study design & analysis

To cater for my lack of knowledge in biological data sampling and analysis, I actually signed up for the 'Wildlife Study Design and Data Analysis' organized by Biodiversity Conservation Society Sarawak

So, this new year, I've decided to take it down a notch and systematically choose my battlefield. Wildlife species data has always been mystery at me. As we all know, biologists hold them close to their hearts to the point of annoyance sometimes (those movies with scientists blindly running after some rare orchids or snakes or something like that really wasn't kidding). Hey...I get it and I totally agree - the data that belongs to the organization has to be treated with utmost confidentiality and all by the experts that collects them. Especially since we all know that they are not something so easily retrieved. Even more so, I optimistically support for the enthusiasm to be extended to their data cleaning and storing too while they're at it. But it doesn't mean I have to like the repercussions. Especially not when someone expects a habitat suitability map from me and I have no data to work with and all I had is a ping-pong game of exchanging jargon in the air with the hopes that the other player gets what you mean cough up something you can work with. Yes...there is not a shred of shame here when I talk about how things work in the world, but it is what it is and I'm not mad. It's just how it works in the challenging world of academics and research. 

To cater for my lack of knowledge in biological data sampling and analysis, I actually signed up for the 'Wildlife Study Design and Data Analysis' organized by

Biodiversity Conservation Society Sarawak (BCSS for short)

or

Pertubuhan Biodiversiti Konservasi Sarawak

It just ended yesterday and I can't say I did not cry internally. From pain and gratitude and accomplishment of the sort. 10 days of driving back and forth between the city center and UNIMAS was worth the traffic shennanigans.  

It is one of those workshops where you really do get down to the nitty-gritty part of understanding probability distribution from scratch; how to use it for your wildlife study data sampling design and analyzing them to obtain species abundance, occupancy or survival. And most importantly, how Bayes has got anything to do with it. I've been hearing and seeing Bayesian stats, methods and network on almost anything that involves data science, R and spatial stats that I am quite piffed that I did not understand a thing. I am happy to inform that now, I do. Suffice to say that it was a bootcamp well-deserved of the 'limited seats' reputation and the certificate really does feel like receiving a degree. It dwindles down to me realizing a few things I don't know:

I did not know that we have been comparing probabilities instead of generating a 'combined' one based on a previous study all these years.

I did not know that Ronald Fisher had such strong influence that he could ban the usage of Bayesian inference by deeming it unscientific.

I did not know that, for Fisher, if the observation cannot be repeated many times and is uncertain, then, the probability cannot be determined - which is crazy! You can't expect to shoot virus into people many times and see them die to generate probability that it is deadly!

I did not know that Bayes theorem actually combines prior probability and the likelihood data you collected on the field for your current study to generate the posterior probability distribution!

I did not know that Thomas Bayes was a pastor and his theory was so opposed to during his time. It was only after Ronald Fisher died that Bayesian inference gain favor especially in medical field. 

I did not know...well...almost anything at all about statistics!

It changed the way I look at statistics basically. But I self-taught myself into statistics for close to 9 years and of course I get it wrong most of the time; now I realize that for the umpph-th time. And for that, I hope the statistics power that be forgives me. Since this boot camp was so effective, I believe it is due to their effort in developing and executing the activities that demonstrates what probability distribution models we were observing. In fact, I wrote down the activities next to the topic just to remember what the deal was. Some of the stuffs covered are basics on Binomial Distribution, Poisson Distribution, Normal/Gaussian Distribution, Posterior probability, Maximum Likelihood Estimate (MLE), AIC, BACI, SECR, Occupancy and Survival probability. Yes...exhausting and I have to say, it wasn't easy. I could listen and distracted by paper falling for a fraction of time just to find myself lost in the barrage of information. What saved me was the fact that we have quizzes that we have to fill in to evaluate our understanding of the topic for the day and discuss them first thing in the next session. Best of all, we were using R with the following packages: wiqid, unmarked, rjags and rasters. Best locations for camera traps installation was discussed as well and all possible circumstances of your data; management and collection itself on the field, were covered rigorously. 

For any of you guys out there who are doing wildlife study, I believe that this boot camp contains quintessential information for you to understand to design your study better. Because once the data is produced, all we can do it dance around finding justification of some common pitfalls that we could've countered quite easily. 

In conclusion, not only that this workshop cast data analysis in a new light for me, but it also helps establishes the correct steps and enunciates the requirements to gain most out of your data. And in my case, it has not only let me understand what could be going on with my pals who go out into the jungle to observe the wildlife first hand, it has also given me ideas on looking for the resources that implements Bayesian statistics/methods on remote sensing and GI in general. Eventhough location analysis was not discussed beyond placing the locations of observation and occasions on the map, I am optimistic in further expanding what I understood into some of the stuff I'm planning; habitat suitability modeling and how to not start image classification from scratch...every single time if that's even possible. 

For more information on more workshops by BCSS or wildlife study design and the tools involved, check out the links below:

Biodiversity Conservation Society Sarawak (BCSS) homepage: https://bcss.org.my/index.htm

BCSS statistical tutorials: https://bcss.org.my/tut/

Mike Meredith's home page: http://mikemeredith.net/

And do check out some of these cool websites that I have referred to for more information as well as practice. Just to keep those brain muscles in loop with these 'new' concepts:

Statistical Rethinking: A Bayesian Course with Examples in R and Stan: https://github.com/rmcelreath/statrethinking_winter2019

Probability Concepts Explained: Introduction by Jonny Brooks-Bartlett: https://towardsdatascience.com/probability-concepts-explained-introduction-a7c0316de465 

Probability Concepts Explained: Maximum Likelihood Estimation by Jonny Brooks-Bartlett: https://towardsdatascience.com/probability-concepts-explained-maximum-likelihood-estimation-c7b4342fdbb1

Probability Concepts Explained: Bayesian Inference for Parameter Estimation by Jonny Brooks-Bartlett 

I'll be posting some of the things I am working on while utilizing the Bayesian stats. I'd love to see yours too!

P/S: Some people prefer to use base R with its simple interface, but if you're the type who works better with everything within your focal-view, I suggest you install RStudio. It's an IDE for R that helps to ease the 'anxiety' of using base R. 

P/S/S: Oh! Oh! This is the most important part of all. If you're using ArcGIS Pro like I do, did you know that it has R-Bridge that can enable the accessibility of R workspace in ArcGIS Pro? Supercool right?! If you want to know more on how to do that, check out this short 2 hour course on how to get the extension in and an example on how to use it: 

Using the R-Bridge: https://www.esri.com/training/catalog/58b5e417b89b7e000d8bfe45/using-the-r-arcgis-bridge/

🧰 Publicly available data

Hunting for spatial data comes naturally now. There seems to be less and less opportunity for doubts when we could attach a pair of coordinates to some places.

For work and hobby, hunting for data take almost half of the usable hours I set aside to execute certain objectives; if not 100%. Although the internet is a vast plain of data, not all of them are usable. The democratization of data is a subject that is to translucent to discuss but to solid to argue with. Thus, with differing opinions, we get different versions of them online. Here are some of the interesting data platforms I manage to scour based on their thematic subject

🌳 Nature and Environment

Delta at Risk - Profiling Risk and Sustainability of Coastal Deltas of the World. I found this while lamenting on how people love asking for data addition into their maps at the eleventh hour. I find their confidence in my skills quite misleading but flattering nonetheless. But it does not make it any less troublesome.

Protected Planet - Discover the world's protected and conserved areas. This platform includes not just data of protected areas, but also other effective area-based conservation measures like ICCAs IUCN listing and as the website claims, it is updated regular via submissions from agencies. So far, I found this platform to be the most convenient since it rounds up all possible conservation-based themes which also includes World Heritage Sites.

Global Forest Change (2000-2020) - The global forest extent change since 2000 to the current year or lovingly referred to as the Hansen data by most forestry RS specialist. This data is updated annually and to be honest, the platforms are literally everywhere. But this platform is legitimate under Earth Engine Apps and you can refer to Google Earth Engine for future data updates to ease your search.

👩‍⚖️ Administrative Data

GADM - Map and spatial data for all countries and their sub-divisions.

🏦 Built-environment Data

OpenStreet Map - This database is the most amazing feat of tech-aware crowdsourcing. A little more than 2 decades ago, some 'experienced' gate-keeping professionals would have refuted its legitimacy within an inch of their lives but OSM has proven that time prevails when it comes to bringing the accessibility and network data into practical use. I am not that adept with downloading from this website so I go directly to a more manual data download. My favorite is the Geofabrik Download but you can also try Planet OSM.

🎮 Other Cool Data

OpenCell ID - Open database platform of global cell towers. Cleaning the data is a nightmare but I think it is just me. I have little patience for cerebral stuff.

So, those are some of the data I managed to dig for personal projects. Hope it helps you guys too!

Floating...in the mid of it

Python: Geospatial Environment Setup (Part 1)

Here’s a quick run down of what you’re supposed to do to prepare yourself to use Python for data analysis.

Install Python ☑

Install Miniconda ☑

Install the basic Python libraries ☑

Create new environment for your workspace

Install geospatial Python libraries

🐍 Installing Python

Let’s cut to the chase. It’s December 14th, 2021. Python 3 is currently at 3.10.1 version. It’s a great milestone for Python 3 but there were heresay of issues concerning 3.10 when it comes to using it with conda. Since we’re using conda for our Python libraries and environment management, we stay safe by installing Python 3.9.5.

Download 👉🏻 Python 3.10.1 if you want to give a hand at some adventurous troubleshooting

Or download 👉🏻 Python 3.9.5 for something quite fuss-free

📌 During installation, don’t forget to ✔ the option Add Python 3.x to PATH. This enables you to access your Python from the command prompt.

Installing Miniconda

As a beginner, you’ll be informed that Anaconda is the easiest Python library manager GUI to implement conda and where it contains all the core and scientific libraries you ever need for your data analysis upon installation. So far, I believe it’s unnecessarily heavy, the GUI isn’t too friendly and I don’t use most of the pre-installed libraries. So after a few years in the darkness about it, I resorted to jump-ship and use the skimped version of conda; Miniconda.

Yes, it does come with the warning that you should have some sort of experience with Python to know what core libraries you need. And that’s the beauty of it. We’ll get to installing those libraries in the next section.

◾ If you’re skeptical about installing libraries from scratch, you can download 👉🏻 Anaconda Individual Edition directly and install it without issues; it takes some time to download due to the big file and a tad bit longer to install.

◾ Download 👉🏻 Miniconda if you’re up to the challenge.

📌 After you’ve installed Miniconda, you will find that it is installed under the Anaconda folder at your Windows Start. By this time, you will already have Python 3 and Anaconda ready in your computer. Next we’ll jump into installing the basic Python libraries necessary for core data analysis and create an environment to house the geospatial libraries.

📚 Installing core Python libraries

Core libraries for data analysis in Python are the followings:

🔺 numpy: a Python library that enables scientific computing by handling multidimensional array objects, or masked objects including matrices and all the mathematical processes involved.

🔺 pandas: enables the handling of ‘relational’ or 'labeled’ data structure in a flexible and intuitive manner. Basically enables the handling of data in a tabular structure similar to what we see in Excel.

🔺matplotlib: a robust library that helps with the visualization of data; static, animated or interactive. It’s a fun library to explore.

🔺 seaborn: another visualization library that is built based on matplotlib which is more high-level and produces more crowd-appealing visualization. Subject to preference though.

🔺 jupyter lab: a web-based user interface for Project Jupyter where you can work with documents, text editors, terminals and or Jupyter Notebooks. We are installing this library to tap into the notebook package that is available with this library installation

To start installing:

1️⃣ At Start, access the Anaconda folder > Select Anaconda Prompt (miniconda3)

2️⃣ An Anaconda Prompt window similar to Windows command prompt will open > Navigate to the folder you would like to keep your analytics workspace using the following common command prompt codes:

◽ To backtrack folder location 👇🏻

◽ Change the current drive, to x drive 👇🏻

◽ Navigate to certain folders of interest e.g deeper from Lea folder i.e Lea\folder_x\folder_y 👇🏻

3️⃣ Once navigated to the folder of choice, you can start installing all of the libraries in a single command as follows:

The command above will enable the simultaneous installation of all the essential Python libraries needed by any data scientists.

💀 Should there be any issues during the installation such as uncharacteristically long installation time; 1 hour is stretching it, press Ctrl + c to cancel any pending processes and proceed to retry by installing the library one by one i.e

Once you manage to go through the installation of the basic Python libraries above, you are half way there! With these packages, you are already set to actually make some pretty serious data analysis. The numpy, pandas and matplotlib libraries are the triple threat for exploratory data analysis (EDA) processes and the jupyter lab library provides the documentation sans coding notebook that is shareable and editable among team mates or colleagues.

Since we’re the folks who like to make ourselves miserable with the spatial details of our data, we will climb up another 2 hurdles to creating a geospatial workspace using conda and installing the libraries needed for geospatial EDA.

If you're issues following the steps here, check out the real-time demonstration of the installations at this link 👇🏻

See you guys in part 2 soon!

📚 Nature in the Heart of Borneo (2020)

Tool: ArcGIS Pro 2.6.1

Technique: Annotation, Labeling and Symbology

A series of maps were created for the book published by WWF-Malaysia and FORMADAT (Forum Masyarakat Adat Dataran Tinggi Borneo) back in 2020 called Nature in the Heart of Borneo.

This book was meant as a guide to some of the natural attractions at Northern parts of Sarawak. If it was clear, Northern Sarawak is where the we have our very own highlanders which consist of primarily the Lundayeh/Lun Bawang, Sa'ban and Kelabit people. Some of the beautiful settlements up in the north that should not be missed are Ba'kelalan and Long Semadoh. They have beautiful homestays and even more beautiful landscapes with trekking activities lined up for tourists. And this is the culmination of ardent passion by my two absolutely wonderful colleagues, Alicia Ng and Cynthia Chin.

Most part of the maps were made using readily available basemap provided by Esri in their Living Atlas. But in entirety, many of the features and details are drawn manually within ArcGIS Pro. Like many other mapmakers out there, the labeling feature is horrendously temperamental and I either end up using annotations instead.

In summary, technically, there are 2 lessons learned here:

1️⃣ Establish concept or pick an idea before you start drawing

A concept of the map and palette should be established at the earliest stage possible. And don't just throw the task of making maps and split them evenly between cartographers. They won't have similar ideas or similar interpretations of the concept. It'll only give you double the pain of creating the maps again from scratch.

2️⃣ Omit borders

If you're making maps for books, don't border trying to make borders and fully utilize the whole layout. In the end, you'll need to export out your maps and they will resize it anyway and it'll compromise the maps you created. As if it wasn't graining enough in the first place, it'll look absolutely microscopic by the time they're done.

With this, I am commencing my submission for the #30DayMapChallenge for 2023 🗺

The categories outlined is similar to that of last year but I am never going to hate this repetition. How can I? It's a basics of making maps and there's so much to learn from the single-word theme.

Any aspiring map-makers out there? Let's share our maps for this wonderful month of November under the #30DayMapChallenge 2023!

[2022] 30 Day Map Challenge -- FAILED

Last year, I participated once again in the 30 Day Map Challenge that was going around in Twitter-ville come November. It is the 3rd attempt at the marathon and 2022 served as a reminder that progressed too despite getting stuck at Day 3 as life caught up with me.

I don't like the idea that I have left the challenge incomplete, again. It was not my priority and I work better with clear goals or visions of expected output. If it does not add to my need to learn something new ...it will be a task bound to head straight to the backburner. Let's resolve to make it a long-term routine instead of a spurt of stress trying to make the deadline.

As a consequence, I am attuning this task into one that actually gives me the benefit out putting into record the techniques and tools I used to make the maps in writing. I believe that will serve more purpose and added value other than visuals. And perhaps, have some stock ready for submission this year instead.

Anyone else participated in this challenge back in November? How did you do and what would you like to do better for the next one? Don't be shy and do drop a word or two.