Opened 4 years ago

Last modified 16 months ago

#1816 new task

[Git] [Step 3] Create .gitattributes for Git code repo

Reported by: Kieran P Owned by:
Priority: Must Have Milestone: Backlog
Component: Build & Packages Keywords:
Cc: Patch:

Description

Create a .gitattributes file for the git code repo. In it, we need to:

  • Force line endings to LF on all files except configs and readmes (since windows users are likely to open those up using notepad which doesn't support LF line endings, only CRLF).
  • (more to come)

Change History (18)

comment:1 Changed 4 years ago by Kieran P

Summary: Create .gitattributes for Git code repo[Git] [Step 3] Create .gitattributes for Git code repo

comment:2 Changed 4 years ago by Kieran P

Milestone: Alpha 13Alpha 14

comment:3 in reply to:  description ; Changed 4 years ago by Ben Brian

Owner: set to Philip Taylor

Replying to k776:

Create a .gitattributes file for the git code repo. In it, we need to:

  • Force line endings to LF on all files except configs and readmes (since windows users are likely to open those up using notepad which doesn't support LF line endings, only CRLF).

Anyone know how to do that? Mostly it needs to work with whatever git client we prefer on Windows, but it should be tested in other environments as well. Line ending consistency might be important for hashing data files for OOS checking and to avoid confusion of the git client wanting to modify line endings of dozens of files.

The relevant settings aren't clearly documented so leaving them up to each user to configure correctly is probably a mistake.

To experiment, try checking out http://git.wildfiregames.com/0ad.git/ (it's ~2GB and unfortunately gets downloaded in one big chunk)

comment:4 Changed 4 years ago by zoot

Hmm. Do we want all files to be binary-identical (for the sake of hashing) across all platforms? Or just data files?

If we want all files to be binary-identical, we could force configs and readmes to be CRLF, while forcing everything else to be LF. Linux and Mac editors can probably cope with the CRLF files (correct me if I'm wrong).

If we do not require all files to be binary-identical, we can let configs and readmes use the line endings native to the user's platform, while forcing everything else to be LF.

Last edited 4 years ago by zoot (previous) (diff)

comment:5 in reply to:  3 ; Changed 4 years ago by zoot

Replying to historic_bruno:

To experiment, try checking out http://git.wildfiregames.com/0ad.git/ (it's ~2GB and unfortunately gets downloaded in one big chunk)

I'm trying to clone this repo, but Git has been stuck on "Cloning into '0ad'..." for 8 hours now. I can't even tell if anything is going on (short of using a packet sniffer!). Could there be a mistake in the server setup? When I clone anything from Github, it always immediately begins reporting progress on the number of objects downloaded.

comment:6 in reply to:  5 Changed 4 years ago by Ben Brian

Replying to zoot:

Replying to historic_bruno:

To experiment, try checking out http://git.wildfiregames.com/0ad.git/ (it's ~2GB and unfortunately gets downloaded in one big chunk)

I'm trying to clone this repo, but Git has been stuck on "Cloning into '0ad'..." for 8 hours now. I can't even tell if anything is going on (short of using a packet sniffer!). Could there be a mistake in the server setup? When I clone anything from Github, it always immediately begins reporting progress on the number of objects downloaded.

I think it took me 5-6 hours and TortoiseGit reported no status or progress until it was finished, the difference may be that it's one huge binary file instead of thousands of smaller files? Annoyingly, the directory doesn't even appear to grow larger in Windows.

comment:7 Changed 4 years ago by zoot

# Force all source and data files to LF, for hash consistency.
*.cpp eol=lf
*.js eol=lf
*.xml eol=lf
*.json eol=lf

# Use native line endings for configuration and plain text files, for easier editing on any platform.
*.cfg text=auto
*.txt text=auto

The above would be my initial proposal for a .gitattributes file. Note that in the repository, all the listed file types will be normalized to LF, but once checked out, *.cfg and *.txt will be converted to the line endings native to the user's platform. Comments?

The .gitattributes man page suggest normalizing everything in one go thusly:

$ rm .git/index     # Remove the index to force git to
$ git reset         # re-scan the working directory
$ git status        # Show files that will be normalized
$ git add -u
$ git add .gitattributes
$ git commit -m "Introduce end-of-line normalization"

comment:8 Changed 4 years ago by Kieran P

Milestone: Alpha 14Alpha 15

comment:9 Changed 4 years ago by Riemer

I wish it already was implemented ;) Currently having some problems applying and checking patches due lf/cr differences (the patch command works, but the git diff just shows the whole file as a diff).

Anyway.. Don't forget to add the *.h files for cpp :)

Also related, maybe we can expand the .gitignore file as well? After compiling under linux, the added files are quite large. Or should I create another ticket for this?

Cheers, Riemer

comment:10 Changed 3 years ago by leper

Milestone: Alpha 15Alpha 16

comment:11 Changed 3 years ago by leper

Milestone: Alpha 16Alpha 17

comment:12 Changed 3 years ago by Ben Brian

Owner: Philip Taylor deleted

comment:13 Changed 3 years ago by leper

Milestone: Alpha 17Alpha 18

comment:14 Changed 2 years ago by Simon Georges

In case it can help, there are here lots of information regarding git and handling big repositories: http://blogs.atlassian.com/2014/05/handle-big-repositories-git/ (especially cloning --depth 1 to only get code, but not the history). The part about the repository with lots of binaries assets can eventually apply too.

comment:15 Changed 2 years ago by Itms

Milestone: Alpha 18Alpha 19

comment:16 Changed 23 months ago by mimo

Milestone: Alpha 19Alpha 20
Priority: Release BlockerMust Have

comment:17 Changed 16 months ago by mimo

moved to backlog as no progress since ages

comment:18 Changed 16 months ago by mimo

Milestone: Alpha 20Backlog
Note: See TracTickets for help on using tickets.