A "Best of the Best Practices" (BOBP) guide to developing in Python.
- "Build tools for others that you want to be built for you." - Kenneth Reitz
- "Simplicity is alway better than functionality." - Pieter Hintjens
dimasmamot
rvilla87
/ MongoDBsparkConnector.scala
Created
August 6, 2017 16:28
Inserting documents in MongoDB with Spark Connector (Dataframe vs Spark Structured Streaming)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // Dataframe (supported) - read 1 file, no streaming | |
| // Step 1, create the Dataframe source | |
| val fileDF = spark | |
| .read // No streaming | |
| .csv("file/file1.csv") | |
| .selectExpr("CAST(key as String)", // more code with other casting... | |
| ) | |
| // Out [1]: fileDF: org.apache.spark.sql.package.DataFrame = [key: string, country: string ... 6 more fields] | |
| // Step 2, insert Dataframe into MongoDB |
sloria
/ bobp-python.md
Last active
September 9, 2025 10:52
A "Best of the Best Practices" (BOBP) guide to developing in Python.
digitaljhelms
/ gist:4287848
Last active
November 23, 2025 17:07
Git/GitHub branching standards & conventions