Skip to content

[SPARK-28734][DOC] Initial table of content in the left hand side bar for SQL doc #25459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
164 changes: 158 additions & 6 deletions docs/_data/menu-sql.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,12 +72,164 @@
url: sql-migration-guide-hive-compatibility.html
- text: SQL Reserved/Non-Reserved Keywords
url: sql-reserved-and-non-reserved-keywords.html
- text: Reference
url: sql-reference.html

- text: SQL Reference
url: sql-ref.html
subitems:
- text: Data Types
url: sql-reference.html#data-types
url: sql-ref-datatypes.html
- text: NaN Semantics
url: sql-reference.html#nan-semantics
- text: Arithmetic operations
url: sql-reference.html#arithmetic-operations
url: sql-ref-nan-semantics.html
- text: SQL Syntax
url: sql-ref-syntax.html
subitems:
- text: Data Definition Statements
url: sql-ref-syntax-ddl.html
subitems:
- text: ALTER DATABASE
url: sql-ref-syntax-ddl-alter-database.html
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you create sub tasks for each page?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gatorsmile Sure.. I will create the sub tasks tomorrow.

- text: ALTER TABLE
url: sql-ref-syntax-ddl-alter-table.html
- text: ALTER VIEW
url: sql-ref-syntax-ddl-alter-view.html
- text: CREATE DATABASE
url: sql-ref-syntax-ddl-create-database.html
- text: CREATE FUNCTION
url: sql-ref-syntax-ddl-create-function.html
- text: CREATE TABLE
url: sql-ref-syntax-ddl-create-table.html
- text: CREATE VIEW
url: sql-ref-syntax-ddl-create-view.html
- text: DROP DATABASE
url: sql-ref-syntax-ddl-drop-database.html
- text: DROP FUNCTION
url: sql-ref-syntax-ddl-drop-function.html
- text: DROP TABLE
url: sql-ref-syntax-ddl-drop-table.html
- text: DROP VIEW
url: sql-ref-syntax-ddl-drop-view.html
- text: TRUNCATE TABLE
url: sql-ref-syntax-ddl-truncate-table.html
- text: REPAIR TABLE
url: sql-ref-syntax-ddl-repair-table.html
- text: Data Manipulation Statements
url: sql-ref-syntax-dml.html
subitems:
- text: INSERT
url: sql-ref-syntax-dml-insert.html
- text: LOAD
url: sql-ref-syntax-dml-load.html
- text: Data Retrieval(Queries)
url: sql-ref-syntax-qry.html
subitems:
- text: SELECT
url: sql-ref-syntax-qry-select.html
subitems:
- text: DISTINCT Clause
url: sql-ref-syntax-qry-select-distinct.html
- text: Joins
url: sql-ref-syntax-qry-select-join.html
- text: ORDER BY Clause
url: sql-ref-syntax-qry-select-orderby.html
- text: GROUP BY Clause
url: sql-ref-syntax-qry-select-groupby.html
- text: HAVING Clause
url: sql-ref-syntax-qry-select-having.html
- text: LIMIT Clause
url: sql-ref-syntax-qry-select-limit.html
- text: Set operations
url: sql-ref-syntax-qry-select-setops.html
- text: Common Table Expression(CTE)
url: sql-ref-syntax-qry-select-cte.html
- text: Subqueries
url: sql-ref-syntax-qry-select-subqueries.html
- text: Query hints
url: sql-ref-syntax-qry-select-hints.html
- text: SAMPLING
url: sql-ref-syntax-qry-sampling.html
- text: WINDOWING ANALYTIC FUNCTIONS
url: sql-ref-syntax-qry-window.html
- text: AGGREGATION (CUBE/ROLLUP/GROUPING)
url: sql-ref-syntax-qry-aggregation.html
- text: EXPLAIN
url: sql-ref-syntax-qry-explain.html
- text: Auxilarry Statements
url: sql-ref-syntax-aux.html
subitems:
- text: Analyze statement
url: sql-ref-syntax-aux-analyze.html
subitems:
- text: ANALYZE TABLE
url: sql-ref-syntax-aux-analyze-table.html
- text: Caching statements
url: sql-ref-syntax-aux-cache.html
subitems:
- text: CACHE TABLE
url: sql-ref-syntax-aux-cache-cache-table.html
- text: UNCACHE TABLE
url: sql-ref-syntax-aux-cache-uncache-table.html
- text: CLEAR CACHE
url: sql-ref-syntax-aux-cache-clear-cache.html
- text: Describe Commands
url: sql-ref-syntax-aux-describe.html
subitems:
- text: DESCRIBE DATABASE
url: sql-ref-syntax-aux-describe-database.html
- text: DESCRIBE TABLE
url: sql-ref-syntax-aux-describe-table.html
- text: DESCRIBE FUNCTION
url: sql-ref-syntax-aux-describe-function.html
- text: DESCRIBE QUERY
url: sql-ref-syntax-aux-describe-query.html
- text: Show commands
url: sql-ref-syntax-aux-show.html
subitems:
- text: SHOW COLUMNS
url: sql-ref-syntax-aux-show-columns.html
- text: SHOW DATABASES
url: sql-ref-syntax-aux-show-databases.html
- text: SHOW FUNCTIONS
url: sql-ref-syntax-aux-show-functions.html
- text: SHOW TABLE
url: sql-ref-syntax-aux-show-table.html
- text: SHOW TABLES
url: sql-ref-syntax-aux-show-tables.html
- text: SHOW TBLPROPERTIES
url: sql-ref-syntax-aux-show-tblproperties.html
- text: SHOW PARTITIONS
url: sql-ref-syntax-aux-show-partitions.html
- text: SHOW CREATE TABLE
url: sql-ref-syntax-aux-show-create-table.html
- text: Configuration Management Commands
url: sql-ref-syntax-aux-conf-mgmt.html
subitems:
- text: SET
url: sql-ref-syntax-aux-conf-mgmt-set.html
- text: RESET
url: sql-ref-syntax-aux-conf-mgmt-reset.html
- text: Resource Management Commands
url: sql-ref-syntax-aux-resource-mgmt.html
subitems:
- text: ADD FILE
url: sql-ref-syntax-aux-resource-mgmt-add-file.html
- text: ADD JAR
url: sql-ref-syntax-aux-resource-mgmt-add-jar.html
- text: Functions
url: sql-ref-functions.html
subitems:
- text: Builtin Functions
url: sql-ref-functions-builtin.html
subitems:
- text: Scalar functions
url: sql-ref-functions-builtin-scalar.html
- text: Aggregate functions
url: sql-ref-functions-builtin-aggregate.html
- text: User defined Functions
url: sql-ref-functions-udf.html
subitems:
- text: Scalar functions
url: sql-ref-functions-udf-scalar.html
- text: Aggregate functions
url: sql-ref-functions-udf-aggregate.html
- text: Arthmetic operations
url: sql-ref-arithmetic-ops.html
5 changes: 4 additions & 1 deletion docs/css/main.css
Original file line number Diff line number Diff line change
Expand Up @@ -196,11 +196,14 @@ a.anchorjs-link:hover { text-decoration: none; }
width: 210px;
float: left;
position: fixed;
overflow-y: scroll;
top: 0;
bottom: 0;
}

.left-menu {
padding: 0px;
width: 199px;
width: 399px;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for leaving a comment in an old PR but @dilipbiswal do you remember why you had to do this? It
Screen Shot 2020-11-12 at 1 16 25 PM

This makes each title look sort of truncated. I was thinking it should be like:

Screen Shot 2020-11-12 at 1 16 46 PM

}

.left-menu h3 {
Expand Down
22 changes: 22 additions & 0 deletions docs/sql-ref-arithmetic-ops.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
layout: global
title: Arithmetic Operations
displayTitle: Arithmetic Operations
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---

Operations performed on numeric types (with the exception of decimal) are not checked for overflow. This means that in case an operation causes an overflow, the result is the same that the same operation returns in a Java/Scala program (eg. if the sum of 2 integers is higher than the maximum value representable, the result is a negative number).
28 changes: 2 additions & 26 deletions docs/sql-reference.md → docs/sql-ref-datatypes.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: global
title: Reference
displayTitle: Reference
title: Data Types
displayTitle: Data Types
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
Expand All @@ -19,10 +19,6 @@ license: |
limitations under the License.
---

* Table of contents
{:toc}

## Data Types

Spark SQL and DataFrames support the following data types:

Expand Down Expand Up @@ -634,25 +630,5 @@ from pyspark.sql.types import *
</td>
</tr>
</table>

</div>

</div>

## NaN Semantics

There is specially handling for not-a-number (NaN) when dealing with `float` or `double` types that
does not exactly match standard floating point semantics.
Specifically:

- NaN = NaN returns true.
- In aggregations, all NaN values are grouped together.
- NaN is treated as a normal value in join keys.
- NaN values go last when in ascending order, larger than any other numeric value.

## Arithmetic operations

Operations performed on numeric types (with the exception of `decimal`) are not checked for overflow.
This means that in case an operation causes an overflow, the result is the same that the same operation
returns in a Java/Scala program (eg. if the sum of 2 integers is higher than the maximum value representable,
the result is a negative number).
22 changes: 22 additions & 0 deletions docs/sql-ref-functions-builtin-aggregate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
layout: global
title: Builtin Aggregate Functions
displayTitle: Builtin Aggregate Functions
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---

**This page is under construction**
22 changes: 22 additions & 0 deletions docs/sql-ref-functions-builtin-scalar.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
layout: global
title: Builtin Scalar Functions
displayTitle: Builtin Scalar Functions
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---

**This page is under construction**
25 changes: 25 additions & 0 deletions docs/sql-ref-functions-builtin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
layout: global
title: Reference
displayTitle: Reference
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---

Spark SQL is a Apache Spark's module for working with structured data.
This guide is a reference for Structured Query Language (SQL) for Apache
Spark. This document describes the SQL constructs supported by Spark in detail
along with usage examples when applicable.
22 changes: 22 additions & 0 deletions docs/sql-ref-functions-udf-aggregate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
layout: global
title: User defined Aggregate Functions (UDAF)
displayTitle: User defined Aggregate Functions (UDAF)
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---

**This page is under construction**
22 changes: 22 additions & 0 deletions docs/sql-ref-functions-udf-scalar.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
layout: global
title: User defined Scalar Functions (UDF)
displayTitle: User defined Scalar Functions (UDF)
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---

**This page is under construction**
Loading