Background
After three decades on the sell-side and buy-side trading desks—from voice brokerage to fully automated cross-asset execution—we moved our alpha pipelines onto Databricks running on AWS. The desk consumes tick data, options Greeks, liquidity pool flows from DeFi AMMs, and factor signals (IC, IR, breadth) streamed into Delta Lake. Traders rely on AI/BI dashboards for intraday risk—PnL explain, VaR drift, and drawdown monitoring during regime shifts. In production, filters must slice by book, instrument, and execution venue. Without flexible filtering, you cannot isolate slippage, track impermanent loss in liquidity pools, or compare strategy performance across bull, bear, and sideways regimes.
Question 1: Dashboard filter types
What filter types are available when building an Databricks AI/BI dashboard?
A Global, page level, and widget level filters
B Global and page level filters
C Page and widget level filters
D Global filters only
Answer: A
Rationale:
A is correct. Databricks AI/BI dashboards support global, page-level, and widget-level filters to enable granular slicing of trading metrics such as PnL, risk, and liquidity exposures across portfolios.
B is incorrect. This omits widget-level filters, which are critical for isolating specific trading signals.
C is incorrect. It excludes global filters, which are needed for consistent filtering across all dashboard views.
D is incorrect. Only global filters are insufficient for detailed analysis at desk or instrument level.
Question 2: Genie natural language interaction
How does AI/BI Genie empower trading desk users to explore data?
A By allowing interaction using natural language chats
B By requiring predefined SQL queries only
C By enforcing Python scripting for all interactions
D By integrating only external BI tools
Answer: A
Rationale:
A is correct. Genie enables natural language interaction, allowing traders to quickly query exposures such as “show drawdown by strategy during high-volatility regimes” without writing SQL.
B is incorrect. Genie's value is reducing reliance on predefined queries.
C is incorrect. Python scripting is not required for basic interaction.
D is incorrect. Genie is native to Databricks and does not rely solely on external BI tools.
Question 3: SQL hierarchy
What is the correct order of the Databricks SQL namespace hierarchy?
A Catalog → Schema → Table
B Schema → Table → Catalog
C Table → Schema → Catalog
D Workspace → Schema → Table
Answer: A
Rationale:
A is correct. Unity Catalog organizes data in a three-level hierarchy: Catalog → Schema → Table, enabling governance across trading datasets such as tick data and factor models.
B is incorrect. The order is reversed.
C is incorrect. Tables do not sit above schema or catalog.
D is incorrect. Workspace is not part of the SQL namespace hierarchy.
Background
On a multi-asset hedge fund desk running systematic strategies, we validate signals using IC, ICIR, and regime-conditioned backtests. Data pipelines ingest market data, DeFi liquidity pool metrics, and execution logs into Lakehouse architecture. Analysts need rapid dashboard refresh and governed access to UC tables for audit and compliance. When publishing dashboards to PMs and risk committees, correctness of permissions, refresh cadence, and dataset lifecycle is critical. From monitoring slippage to detecting alpha decay, dashboards act as real-time control towers. Efficient dataset management ensures traders react before liquidity dries up or volatility regimes shift, avoiding catastrophic drawdowns.
Question 4: Dashboard sharing prerequisite
What must a data analyst do before sharing a dashboard with stakeholders?
A Publish the dashboard
B Clone the draft version
C Configure refresh schedules
D Validate SQL queries
Answer: A
Rationale:
A is correct. Publishing the dashboard makes it available to stakeholders such as portfolio managers and risk teams.
B is incorrect. Cloning does not make the dashboard accessible.
C is incorrect. Refresh schedules are important but not required before sharing.
D is incorrect. Query validation is useful but not a prerequisite for sharing.
Question 5: Dataset creation tab
Which tab allows defining a dataset using SQL in Databricks dashboards?
A Data
B Genie
C Filters
D Visualizations
Answer: A
Rationale:
A is correct. The Data tab is where SQL queries define datasets for dashboards.
B is incorrect. Genie is used for conversational queries, not dataset definition.
C is incorrect. Filters modify views, not datasets.
D is incorrect. Visualizations display data but do not define datasets.
Question 6: Permissions governance
How are permissions handled when using a Unity Catalog table in dashboards?
A Unity Catalog governance controls permissions
B Dashboard-level permissions override all
C Permissions are ignored
D Full access is granted by default
Answer: A
Rationale:
A is correct. Unity Catalog enforces centralized governance, ensuring compliance with trading desk data policies.
B is incorrect. Dashboard permissions do not override UC governance.
C is incorrect. Permissions are strictly enforced.
D is incorrect. Access must be explicitly granted.
Background
Running real-time risk and PnL dashboards on AWS-backed Databricks requires tight integration between compute and SQL warehouses. On the desk, we run intraday factor recalcs, VaR shocks, and liquidity stress scenarios. Genie Spaces must be backed by compute resources capable of handling high concurrency queries. Traders frequently onboard new datasets—from order book depth to DeFi yield signals—and must explore them before building visualizations. In fast markets, publishing stale dashboards is equivalent to trading blind; data freshness and compute coupling define whether you catch a regime shift or miss it entirely.
Question 7: Genie space compute
Which compute resource must be associated when creating a Genie Space?
A SQL warehouse
B All-purpose cluster
C External location
D Metastore
Answer: A
Rationale:
A is correct. SQL warehouses power query execution for BI and Genie interactions.
B is incorrect. All-purpose clusters are not required for Genie Spaces.
C is incorrect. External locations relate to storage, not compute.
D is incorrect. Metastore handles metadata, not execution.
Question 8: First step for new dataset
What is the first step when enhancing a dashboard with a new dataset?
A Locate and explore the dataset
B Create visualizations
C Publish dashboard
D Create Genie Space
Answer: A
Rationale:
A is correct. Analysts must first understand the dataset—similar to validating signal quality before deploying trading strategies.
B is incorrect. Visualization comes after understanding data.
C is incorrect. Publishing occurs later.
D is incorrect. Genie Space is not required for dataset onboarding.
Question 9: Refresh mechanism
How is dashboard data updated in Databricks?
A Refreshed on demand or on a schedule
B Automatically by Unity Catalog
C Only by manual SQL execution
D Only when cloning dashboard
Answer: A
Rationale:
A is correct. Data refresh can be scheduled or triggered, ensuring traders view up-to-date risk and performance metrics.
B is incorrect. Unity Catalog does not refresh data.
C is incorrect. Manual refresh is not the only method.
D is incorrect. Cloning does not refresh datasets.
Background
In quantitative research, we constantly optimize queries to investigate why execution costs spike or why IC deteriorates. Using Photon engine, caching, and Z-ordering, we accelerate queries on billions of rows—tick data, options chains, and DeFi transaction logs. On AWS-backed Databricks, performance tuning directly impacts time-to-decision. During volatile markets, delays in query execution can lead to slippage or missed arbitrage. The Lakehouse architecture allows unified processing of batch and streaming data, which is critical when managing hybrid CeFi and DeFi portfolios across multiple liquidity venues.
Question 10: Query performance optimization
What helps optimize query performance in Databricks?
A Photon engine, caching, and clustering
B Increasing number of dashboards
C Using only notebooks
D Disabling autoscaling
Answer: A
Rationale:
A is correct. Photon, caching, and clustering improve query performance, critical for real-time analytics.
B is incorrect. Dashboards do not improve performance.
C is incorrect. Notebooks are not a performance optimization tool.
D is incorrect. Autoscaling improves efficiency, not performance reduction.
Question 11: Unity Catalog purpose
What is the main purpose of Unity Catalog?
A Centralized governance and lineage
B Compute scaling
C Query federation only
D Dashboard rendering
Answer: A
Rationale:
A is correct. Unity Catalog provides governance, lineage, and access control across data assets.
B is incorrect. Compute scaling is unrelated.
C is incorrect. Federation is only one feature.
D is incorrect. UC is not a visualization tool.
Question 12: Query federation
What is a key property of query federation?
A Data remains in place and is read-only
B Data is copied into Databricks
C Supports write operations
D Requires ETL pipelines
Answer: A
Rationale:
A is correct. Federation queries external systems without moving data, ideal for live trading data integration.
B is incorrect. No data copying occurs.
C is incorrect. Federation is read-only.
D is incorrect. No ETL is required.
Background
On a cross-asset systematic trading desk, we rely on Databricks Lakehouse running on AWS to unify ingestion, transformation, and orchestration of both CeFi and DeFi data. We stream order book depth, DeFi liquidity pool flows, and volatility surfaces into Delta Lake, then use Lakeflow pipelines for ETL. In high-volatility regimes, latency and cost matter—autoscaling, job clusters, and proper instance selection directly impact PnL. From experience managing capital through multiple crises, inefficient compute or poor storage design can erode alpha faster than slippage. Modern data engineering discipline—batch + streaming unification, schema evolution, and governance—is as critical as signal quality on the trading desk.
Question 13: Lakeflow ingestion
What component is responsible for data ingestion in the Databricks platform?
A Lakeflow Connect
B Lakeflow Jobs
C Unity Catalog
D Photon engine
Answer: A
Rationale:
A is correct. Lakeflow Connect is designed for ingestion, bringing structured and streaming trading data—such as exchange feeds or DeFi pools—into the Lakehouse. This aligns with modern data engineering pipelines where ingestion is a dedicated layer handling source connectivity and incremental loading.
B is incorrect. Lakeflow Jobs orchestrates workflows, not ingestion.
C is incorrect. Unity Catalog governs access and metadata, not ingestion.
D is incorrect. Photon improves query performance but does not ingest data.
Question 14: Transformation layer
Which Databricks component handles transformation logic in pipelines?
A Spark Declarative Pipelines
B Lakeflow Connect
C SQL Warehouse
D External location
Answer: A
Rationale:
A is correct. Spark Declarative Pipelines handle transformations for both batch and streaming, enabling consistent ETL logic for trading signals and analytics. This reflects data engineering best practice of separating transformation logic from ingestion and orchestration.
B is incorrect. Lakeflow Connect handles ingestion only.
C is incorrect. SQL Warehouse executes queries but does not manage pipeline transformations.
D is incorrect. External locations manage data access, not transformation logic.
Question 15: Orchestration
What component is used to orchestrate workflows in Databricks?
A Lakeflow Jobs
B Unity Catalog
C Photon engine
D Delta Sharing
Answer: A
Rationale:
A is correct. Lakeflow Jobs orchestrates pipelines, scheduling and running ETL workflows critical for trading desk operations like nightly risk recalculation or intraday signal refresh. This aligns with data engineering principles of workflow orchestration and dependency management.
B is incorrect. Unity Catalog handles governance.
C is incorrect. Photon is a compute engine optimization.
D is incorrect. Delta Sharing is for data sharing, not orchestration.
Background
In real trading environments, cost discipline is embedded into infrastructure decisions. From my experience managing large-scale quant systems, compute inefficiency directly reduces Sharpe ratio. Databricks on AWS enables granular cost controls—autoscaling clusters, job clusters for ephemeral workloads, and tagging for cost attribution. When running backtests across multiple regimes or recalculating IC/IR signals, compute bursts are needed but must not persist. Serverless compute and correct cluster selection ensure optimal cost-performance. Data engineering practices emphasize elasticity and workload isolation—core principles for scalable and cost-efficient trading analytics platforms.
Question 16: Cost optimization strategy
Which approach BEST reduces compute cost in Databricks workloads?
A Use job clusters and autoscaling
B Use only all-purpose clusters
C Disable autoscaling
D Increase cluster size permanently
Answer: A
Rationale:
A is correct. Job clusters and autoscaling ensure compute resources are used only when needed, aligning with data engineering principles of elasticity and cost efficiency.
B is incorrect. All-purpose clusters run continuously and increase cost.
C is incorrect. Disabling autoscaling leads to inefficient resource usage.
D is incorrect. Increasing cluster size permanently increases cost unnecessarily.
Question 17: Performance optimization
Which technique improves performance for large datasets in Databricks?
A Z-order clustering
B Increasing dashboard count
C Using only notebooks
D Disabling caching
Answer: A
Rationale:
A is correct. Z-order clustering improves data locality and query efficiency, a key data engineering optimization technique for large datasets like tick-level market data.
B is incorrect. Dashboards do not impact storage/query performance.
C is incorrect. Notebooks do not inherently optimize performance.
D is incorrect. Disabling caching reduces performance.
Question 18: External location setup
What is the correct sequence to configure an external location?
A Create catalog → create connection → create storage credential
B Create workspace → create cluster → create connection
C Create IAM role → create subnet → create workspace
D Create dataset → create dashboard → publish
Answer: A
Rationale:
A is correct. External location setup follows governance-first design: catalog, connection, then credential, aligning with structured data access patterns in data engineering systems.
B is incorrect. These steps relate to compute setup, not storage access.
C is incorrect. Infrastructure setup is unrelated to external locations.
D is incorrect. These are analytics steps, not storage configuration.
Background
On global macro and quant desks, we integrate multi-cloud data—AWS for equities, Azure for risk systems, and GCP for alternative datasets. Databricks abstracts these differences, but architects must understand identity, storage, and networking models. From a practitioner perspective, misconfiguring access models (IAM roles vs service accounts) can halt data pipelines. The Lakehouse enforces consistent governance across clouds via Unity Catalog. Data engineering best practices demand separation of storage, compute, and identity while ensuring secure, scalable integration—critical when trading systems rely on federated data sources without ETL duplication.
Question 19: Cross-cloud storage access
How does AWS Databricks typically access S3 storage?
A Using IAM roles
B Using service accounts
C Using Access Connector
D Using user credentials
Answer: A
Rationale:
A is correct. AWS Databricks uses IAM roles for secure access to S3, aligning with cloud-native data engineering patterns for identity-based access control.
B is incorrect. Service accounts are used in GCP.
C is incorrect. Access Connector is used in Azure.
D is incorrect. User credentials are not scalable or secure.
Question 20: Query federation architecture
What is the primary benefit of query federation in Databricks?
A Query external systems without moving data
B Copy external data into Delta Lake
C Enable write operations to external systems
D Replace ETL pipelines
Answer: A
Rationale:
A is correct. Query federation enables direct querying of external systems, reducing data duplication and aligning with modern data engineering principles of minimizing data movement. This is critical for trading desks needing real-time access to external liquidity or pricing systems.
B is incorrect. Federation does not copy data.
C is incorrect. Federation is read-only.
D is incorrect. It complements, not replaces, ETL pipelines.