[Git] [Step 3] Create .gitattributes for Git code repo

[Kieran P]

Create a .gitattributes file for the git code repo. In it, we need to:

• Force line endings to LF on all files except configs and readmes (since windows users are likely to open those up using notepad which doesn't support LF line endings, only CRLF).
• (more to come)

[historic_bruno]

Replying to k776:

Create a .gitattributes file for the git code repo. In it, we need to:

• Force line endings to LF on all files except configs and readmes (since windows users are likely to open those up using notepad which doesn't support LF line endings, only CRLF).

Anyone know how to do that? Mostly it needs to work with whatever git client we prefer on Windows, but it should be tested in other environments as well. Line ending consistency might be important for hashing data files for OOS checking and to avoid confusion of the git client wanting to modify line endings of dozens of files.

The relevant settings aren't clearly documented so leaving them up to each user to configure correctly is probably a mistake.

To experiment, try checking out ​http://git.wildfiregames.com/0ad.git/ (it's ~2GB and unfortunately gets downloaded in one big chunk)

[zoot]

Hmm. Do we want all files to be binary-identical (for the sake of hashing) across all platforms? Or just data files?

If we want all files to be binary-identical, we could force configs and readmes to be CRLF, while forcing everything else to be LF. Linux and Mac editors can probably cope with the CRLF files (correct me if I'm wrong).

If we do not require all files to be binary-identical, we can let configs and readmes use the line endings native to the user's platform, while forcing everything else to be LF.

[zoot]

Replying to historic_bruno:

To experiment, try checking out ​http://git.wildfiregames.com/0ad.git/ (it's ~2GB and unfortunately gets downloaded in one big chunk)

I'm trying to clone this repo, but Git has been stuck on "Cloning into '0ad'..." for 8 hours now. I can't even tell if anything is going on (short of using a packet sniffer!). Could there be a mistake in the server setup? When I clone anything from Github, it always immediately begins reporting progress on the number of objects downloaded.

[historic_bruno]

Replying to zoot:

Replying to historic_bruno:

To experiment, try checking out ​http://git.wildfiregames.com/0ad.git/ (it's ~2GB and unfortunately gets downloaded in one big chunk)

I'm trying to clone this repo, but Git has been stuck on "Cloning into '0ad'..." for 8 hours now. I can't even tell if anything is going on (short of using a packet sniffer!). Could there be a mistake in the server setup? When I clone anything from Github, it always immediately begins reporting progress on the number of objects downloaded.

I think it took me 5-6 hours and TortoiseGit reported no status or progress until it was finished, the difference may be that it's one huge binary file instead of thousands of smaller files? Annoyingly, the directory doesn't even appear to grow larger in Windows.

[zoot]

# Force all source and data files to LF, for hash consistency.
*.cpp eol=lf
*.js eol=lf
*.xml eol=lf
*.json eol=lf

# Use native line endings for configuration and plain text files, for easier editing on any platform.
*.cfg text=auto
*.txt text=auto

The above would be my initial proposal for a .gitattributes file. Note that in the repository, all the listed file types will be normalized to LF, but once checked out, *.cfg and *.txt will be converted to the line endings native to the user's platform. Comments?

The ​.gitattributes man page suggest normalizing everything in one go thusly:

$ rm .git/index     # Remove the index to force git to
$ git reset         # re-scan the working directory
$ git status        # Show files that will be normalized
$ git add -u
$ git add .gitattributes
$ git commit -m "Introduce end-of-line normalization"

[Riemer]

I wish it already was implemented ;) Currently having some problems applying and checking patches due lf/cr differences (the patch command works, but the git diff just shows the whole file as a diff).

Anyway.. Don't forget to add the *.h files for cpp :)

Also related, maybe we can expand the .gitignore file as well? After compiling under linux, the added files are quite large. Or should I create another ticket for this?

Cheers, Riemer

[Simon Georges]

In case it can help, there are here lots of information regarding git and handling big repositories: ​http://blogs.atlassian.com/2014/05/handle-big-repositories-git/ (especially cloning --depth 1 to only get code, but not the history). The part about the repository with lots of binaries assets can eventually apply too.

[mimo]

moved to backlog as no progress since ages

[hamletmun]

​https://devblogs.microsoft.com/commandline/extended-eol-in-notepad/

Introducing extended line endings support in Notepad

Starting with the current Windows 10 Insider build, Notepad will support Unix/Linux line endings (LF), Macintosh line endings (CR), and Windows Line endings (CRLF) as usual. New files created within Notepad will use Windows line ending (CRLF) by default, but it will now be possible to view, edit, and print existing files, correctly maintaining the file’s current line ending format.

May 8th, 2018