A Guide to Listing Files and Directories with (Py)Spark, or How To Summon the Beast
Different methods for traversing file-systems with PySpark. From the mundane to the occult.
spark
A collection of 3 posts
Increase PySpark's JDBC parallelism through predicates
Increase the parallelism of loading data through JDBC with Spark using predicates on non-numeric columns.
Loading Data Into Neptune With Spark
Some tips on loading data into AWS Neptune with Apache Spark